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An  informal  photograph  of  the  joint  Chief  of  Stuff.  Left  to  right:  General  Alfred  M.  Gray,  USMC;  Admiral  Carlisle  AH  Trost,  USN ;  General  Larry  D.  Welch, 
USAF;  General  Carl  Edward  Vuono,  USA ;  Admiral  William  J.  Crowe,  fr.}  USN,  chairman;  and  General  Robert  T.  Hems,  USAF. 


Defense  acquisition  process  reform  is  a  cottage  in¬ 
dustry  in  Washington,  D.C.,  and  1986  was  a 
banner  year.  Two  events  occurred  which  have  changed  the 
manner  in  which  the  defense  acquisition  bureaucracy 
functions — passage  of  the  Defense  Reorganization  Act  of 
1986  and  issuance  of  National  Security  Decision  Directive 
(NSDD)  219. 

These  two  events  profoundly  affected  the  Office  of  the 
Secretary  of  Defense  (OSD),  the  Services,  and  the  Organiza¬ 
tion  of  the  Joint  Chiefs  of  Staff  (OJCS).  This  article  defines 
the  role  the  OJCS  now  plays  and  can  be  expected  to  play 
in  the  acquisition  process  and  summarizes  the  potential  im¬ 
pact  on  the  program  manager. 

The  OJCS  was  reorganized  in  1986,  and  so  terms  of 
reference  should  be  redefined  even  for  those  who  have 
served  there.  The  OJCS  consists  of  the  Chairman,  Joint 
Chiefs  of  Staff  (CJCS),  the  Vice  Chairman,  Joint  Chiefs  of 
Staff  (VCJCS),  and  their  personal  staffs;  the  Joint  Chiefs  of 
Staff  (JCS);  the  Joint  Staff,  and  the  individual  agencies  and 
boards  reporting  thereto.  The  Joint  Staff  now  supports  the 
CJCS  in  his  role  as  the  independent  military  advisor  to  the 
Secretary  of  Defense  (SECDEF),  the  National  Security  Coun¬ 
cil,  and  President. 


Vice  Chairman,  Joint  Chiefs  of  Staff,  is  a  new  position 
created  by  the  Reorganization  Act  with  responsibilities  de¬ 
fined  by  the  CJCS  and  approved  by  the  SECDEF.  The 
VCJCS  is  the  second-ranking  military  member  of  the  Depart¬ 
ment  of  Defense  and,  by  CJCS  option,  the  focal  point  for 
OJCS  involvement  in  systems  acquisition. 

In  general,  the  VCJCS  is  responsible  for  seeing  that  CJCS 
goals— improving  the  way  joint  system  requirements  are 
identified,  justified,  analyzed,  and  satisfied  and  improving 
the  way  joint  programs  are  managed — are  achieved.  Dur¬ 
ing  the  long-term,  the  VCJCS  will  be  the  tJiread  of  continuity 
from  CINC  requirements  to  Service  Program  Objective 
Memorandums. 

These  goals  are  not  new.  In  1984,  the  Defense  Science 
Board  identified  the  need  for  a  senior  group  tc  look  after 
the  requirements  for,  and  management  of,  joint  programs. 
Shortly  thereafter,  the  JCS  chartered  the  Joint  Requirements 
and  Management  Board  (JRMB)  which  did  yeoman  service 
during  its  first  two  years  of  operation.  The  original  JRMB 
consisted  of  the  Vice  Chiefs  of  the  Army  and  Air  Force,  The 
Vice  Chief  of  Naval  Operations  the  Assistant  Commandant 
of  the  Marine  Corps,  and  the  Director,  Joint  Staff.  Chair¬ 
manship  was  rotated  annually  among  Service  members. 
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More  than  $4.5  billion  in  cost 
avoidance  were  documented  as  a  result 
of  JRMB  efforts  to  identify  oppor¬ 
tunities  for  jointness  in  systems 
acquisition. 

Pragmatically,  JRMB  efforts  focused 
more  on  the  reduction  of  duplication 
of  effort  in  acquisition  programs 
underway  than  on  melding  individual 
Service  requirements  into  joint  ones. 
Whatever  its  focus,  the  JRMB  was  a 
quiet,  insistent,  and  effective  force  for 
giving  the  taxpayers  more  for  their 
dollars;  so  effective,  in  fact,  that  the 
Packard  Commission  seized  upon  the 
JRMB  name  to  replace  the  name  (and 
to  some  extent,  the  function)  of  the 
Defense  Systems  Acquisition  Review 
Council  (DSARC).  Accordingly,  in 
June  1986,  the  original  JRMB  was 
renamed  the  Joint  Requirements  Over¬ 
sight  Council  (JROC);  its  charter  was 
not  changed  at  this  time. 

Before  1984,  the  OJCS  usually  pro¬ 
vided  a  senior  representative  to 
DSARC  deliberations.  Participation  in 
the  acquisition  decision  process  was 
largely  ad  hoc,  however,  because  there 
was  no  bureaucratic  infrastructure 
within  the  Joint  Staff  to  support  any 
greater  level  of  involvement.  Such  in¬ 
volvement  was  not  envisioned  when 
OJCS  was  created  and  staffed.  This  has 
given  rise  to  a  definitely  outdated  and 
unnecessarily  broad  criticism  within 
the  acquisition  bureaucracy  that  "the 
JCS  never  met  a  requirement  it  didn't 
like." 

This  situation  was  in  remedy  during 
the  same  period  as  the  early  JROC  ac¬ 
tivity.  The  Strategic  Plans  and 
Resource  Analysis  Agency  (SPRAA), 
created  at  the  behest  of  the  Congress, 
gave  the  CJCS  something  of  a  program 
analysis  and  evaluation  capability.  The 
SPRAA  was  staffed  with  officers  who 
understood  both  the  acquisition  pro¬ 
cess  and  the  Programming,  Planning, 
and  Budgeting  System  (PPBS).  The 
OJCS  attendees  at  DSARC  meetings 
began  to  arrive  more  fully  prepared  to 
deal  with  the  issues,  but  a  continuous 
thread  to  OJCS  involvement  in  the 
process  was  absent. 


When  the  first  VCJCS,  General 
Robert  T.  Herres  of  the  U.  S.  Air  Force 
arrived,  many  changes  were  imminent. 
The  NSDD  219,  implementer  for  the 
Packard  Commission  recommenda¬ 
tions,  had  just  been  received  for  action 
and  the  Joint  Staff  had  been  restruc¬ 
tured  in  anticipation  of  the  Chairman's 
changing  role.  Mr.  Richard  Godwin, 
had  just  assumed  the  newly  created 
position  of  Under  Secretary  of  Defense 
for  Acquisition  (USD(A)). 


nn 

JL  he  DODD 
5000.49  created  the 
Defense  Acquisition 
Board  and  ten  com¬ 
mittees  were  created 
to  replace  the  126 
boards  and  councils 
that  had  carved  out 
their  own  niches  in 
the  bureaucratic 
cliff. 


Considerable  activity  ensued  in  the 
weeks  and  months  following  their  ar¬ 
rivals.  The  DOD  Directive  5000.1  and 
DODI  5000.2  were  rewritten  and 
staffed.  The  DODD  5000,49  created 
the  Defense  Acquisition  Board  (DAB), 
(a  successor  to  the  JRMB,  aka 
DSARC),  and  ten  DAB  committees 


were  created  to  replace  the  126  boards 
and  councils  that  had  carved  out  their 
own  niches  in  the  bureaucratic 
cliff. 

With  respect  to  acquisition,  the 
VCJCS  was  specifically  given  respon¬ 
sibility  for:  (1)  bringing  a  strategic 
viewpoint  to  acquisition  decision¬ 
making,  (2)  working  for  reduced 
duplication  of  capabilities  and  increas¬ 
ed  interoperability  based  on  opera¬ 
tional  lessons  learned,  (3)  chairing  the 
JROC,  (4)  vice  chairing  the  DAB,  (5) 
representing  the  OJCS  on  the  Nuclear 
Weapons  Council  (6)  acting  as  CJCS 
spokesman  on  PPBS  matters  and  as  his 
representative  to  the  Defense 
Resources  Board  (DRB);  and,  (7)  act¬ 
ing  as  CINC  representative  on  acquisi¬ 
tion  and  requirements  matters. 

Strategic  Viewpoint 

What  we  acquire  to  equip  the  com¬ 
ponents  must  be  consistent  with  who, 
where,  what,  when,  and  how  we  plan 
to  use  our  military  forces  to  achieve 
national  strategic  goals.  This  seems  to 
be  a  first  principle  of  defense  acquisi¬ 
tion,  but  there  has  been  a  growing  feel¬ 
ing  that  our  arsenal  was  in  charge  of 
our  strategy. 

The  VCJCS  has  been  charged  with 
ensuring  that  the  horse  is,  in  fact,  in 
front  of  the  cart.  This  is  especially  im¬ 
portant  as  we  deal  with  the  concept  of 
fiscally  constrained  strategies.  We  can¬ 
not  afford  to  be  spending  scarce 
resources  on  development  programs 
which  may  do  more  to  reinforce  the  in¬ 
stitutional  identity  of  the  developer 
than  contribute  to  the  national 
defense. 

Reduced  Duplication  and 
Increased  Interoperability 

Frankly,  jointness  is  tough.  It's 
tough  on  the  battlefield,  and  it's  tough 
in  the  program  office.  But  just  as 
frankly,  there  is  only  one  war — no 
land  war,  no  air  war,  no  sea  war- 
just  one  war.  Every  unnecessarily 
duplicated  capability  reduces  the 
resources  available  to  deal  with  other 
problems.  Every  system  which  is  insuf¬ 
ficiently  interoperable  is  a  boon  to  the 


DISCLAIMER:  This  article  represents  the  v  ten’s  of  the  author  and  does  not  necessarily  reflect  the  official  views  of  the  Organization 
of  the  Joint  Chiefs  of  Staff  or  the  Department  of  Defense.  The  article  is  based  on  the  author's  presentations  to  the  DSMC  Executive 
Refresher  and  Executive  Management  Courses,  and  has  been  reviewed  by  the  appropriate  Joint  Staff  offices. 
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bad  guys,  and  a  bull's-eye  on  cur  own 
instep.  The  Joint  Staff  will  be  ag¬ 
gressively  supporting  the  Vice  Chair¬ 
man's  efforts  to  make  sure  that  these 
forms  of  jointness  are  achieved.  An  en¬ 
tire  Joint  Staff  Directorate,  the  Direc¬ 
tor  for  Operational  Plans  and  Inter¬ 
operability,  J-7,  has  been  established 
to  seek  after,  define  and  monitor  the 
implementation  of  means  to  improve 
our  combatant  forces  interoperability. 
The  J-7  also  provides  staff  support  for 
the  JROC.  The  Director  for  Com¬ 
mand,  Control,  and  Communications 
Systems,  J-6,  has  been  the  focal  point 
for  interoperability  for  C3  systems  and 
will  continue  to  be. 


Joint  Requirements 
Oversight  Council 


As  JROC  Chairman,  the  VCJCS  has 
had  the  Council  rechartered  to  move 
its  deliberations  further  "upstream" 
toward  Milestone  "0"  to  impact  con¬ 
cepts  as  they  develop  rather  than  mere¬ 
ly  adjust  programs  as  they  are  ex¬ 
ecuted.  Intuitively,  it  is  better  to  have 
jointness  considered  early  and  applied 
prudently  than  it  is  to  have  it  forced 
into  a  program  after  it  has  been  struc¬ 
tured.  Two  mechanisms  are  necessary 
to  make  the  JROC  effective  in  this  role. 
One  deals  with  Service  development 
activities  and  the  other  with  CINC 
capability  requirements. 


The  annual  Joint  Potential  Review 
and  Designation  Process  is  the 
mechanism  dealing  with  Service 
development  activities  and  produces 
the  Joint  Potential  Designation  List 
(JPDL).  The  intent  of  the  process  is  to 
stimulate  communication  among  the 
Services.  All  new  Service  and 
specified/unified  command  require¬ 
ments  documents,  all  research  and 
development  (R&D)  programs  ap¬ 
proved  for  initiation  and  included  in 
the  Service  Program  Objective 
Memorandums,  and  all  programs  fac¬ 
ing  a  Milestone  I  or  II  review  in  the 
next  fiscal  year  are  subject  to  review. 
All  four  Services  must  review,  com¬ 
ment  on,  and  assign  Joint  Potential 
Designators  (JPD)  to  all  candidate  pro¬ 
grams.  The  JPDs  are  required  for  all 
programs  from  acquisition  category  I 
through  IV  and  are  as  follows: 


— ]oint,  a  potential  for  joint  R&D  pro¬ 
gram  management  and/or  joint  pro¬ 
curement  exists. 
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☆  ☆  ☆  ☆ 


—  Interoperating,  joint  program 
management  is  inappropriate,  but  a 
potential  for  joint  operation  or  joint 
system  interface  exists. 


— Independent ,  no  potential  for  other 
Service  use  or  joint  systems 
development. 

The  mechanism  for  dealing  with 
CINC  capability  requirements  is  under 
development.  It  has  much  in  common 
with  the  U.S.  Army  Concept  Based  Re¬ 
quirements  System.  The  intent  is  to 


We 


e  cannot 


afford  to  be 
spending  scarce 


resources  on 


development  pro¬ 
grams  which  may 


do  more  to  rein¬ 


force  the  institu¬ 


tional  identity  of 
the  developer 


than  contribute 


to  the  national 


defense. 


provide  a  method  for  developing  con¬ 
ceptual  approaches  to  solving  CINC 
problems  before  component  respon¬ 
sibilities  are  defined  rather  than  hav¬ 
ing  component  involvement  dictate  the 
concept. 


Both  of  these  mechanisms  will  pro¬ 
vide  information  central  to  DAB 
deliberations,  and  the  Vice  Chairman, 


JCS,  is  uniquely  positioned  to  facilitate 
JROC-DAB  communication  and  inter¬ 
action,  The  ultimate  goal  is  for  the 
Joint  Staff  to  have  a  well-defined  posi¬ 
tion  on  the  requirements  for,  and 
relative  merits  of,  every  DAB  level 
program. 


Defense  Acquisition  Board 


With  the  VCJCS,  as  the  DAB  Vice 
Chairman,  articulating  Joint  Staff  posi¬ 
tions  which  balance  "jointness"  and 
marginal  return  against  prudent 
duplication  of  capability,  the  DAB  car. 
properly  focus  on  how  we  buy  what 
we  buy;  there  are  other  forums  for 
deciding  what  we  buy.  This  means  that 
the  Joint  Staff  will  be  focused  on  keep¬ 
ing  non-military  influences  from  im¬ 
properly  changing  the  goals  of  a 
development  program  from  what  was 
intended  when  the  requirement  was 
validated. 


The  OJCS  is  not  there  to  side  with 
OSD  against  a  Service,  nor  with  a  Ser¬ 
vice  against  OSD.  The  OJCS  can  be 
expected  to  take  an  independent  posi¬ 
tion  on  each  program,  and  that  posi¬ 
tion  will  be  rooted  in  linking  strategy 
to  plans,  plans  to  requirements,  and  re¬ 
quirements  to  programs. 


This  means  that  the  Joint  Staff  will 
participate  fully  in  DAB  Committee 
activities.  The  current  ten  committees 
were  created  to  replace  an  anarchic  ap¬ 
proach  to  program  oversight.  Each 
committee  has  a  Joint  Staff  Flag  or 
General  Officer  member.  Because  the 
Joint  Staff  has  limited  acquisition  pro¬ 
fessional  resources  these  committee 
responsibilities  have  been  "matrixed 


(ft  is  useful  to  remember  when  deal¬ 
ing  with  the  1,400  member  Joint  Staff 
that  fewer  than  140  have  an  occasional 
contact  with  acquisition  issues,  some 
14  deal  with  acquisition  daily,  and  6 
or  so  are  acquisition  management 
professionals.) 


The  major  workload  in  support  of 
DAB  activities  is  handled  by  the  Direc¬ 
tor  for  Force  Structure,  Resource,  and 
Assessment  Directorate,  J-8,  and  the 
J-7.  The  J-7  oversees  programs  up  to 
the  draft  Systems  Concept  Paper,  and 
J-8  takes  over  from  there  through 
system  retirement.  The  J-7  is  respon¬ 
sible  for  seeing  that  the  tie  between 
plans  and  requirements  is  bound  tight- 
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ly,  and  the  J-8  makes  sure  that  the  re¬ 
quirement  is  adhered  to  as  the  program 
proceeds.  Some  of  the  other  Joint  Staff 
Directors  have  similar  responsibilities 
for  certain  classes  of  systems,  such  as 
Chemical  Weapons  and  Nuclear 
Warheads  (J-5). 

The  VCJCS  role  in  the  Nuclear 
Weapons  Council  (NWC)  is  similar  to 
his  role  in  the  DAB,  but  his  support  for 
NWC  activities  comes  almost  ex¬ 
clusively  from  the  Director  for 
Strategic  Plans  and  Policy,  J-5. 

Planning,  Programming,  and 
Budgeting 

Once  the  DAB  has  determined  that 
a  given  program's  risks  and  acquisition 
strategy  are  consistent  with  program 
goals,  the  Defense  Resources  Board 
(DRB)  determines  whether,  and  at 
what  level,  the  program  will  be 
funded.  Though  not  often  thought  of 
in  these  terms,  it  is  actually  the  DRB 
which  provides  Milestone  "0"  ap¬ 
proval,  not  the  DAB.  As  the  CJCS 
representative  to  the  DRB,  the  Vice 
Chairman  will  be  working  to  ensure 
that  DRB  decisions  underwrite  stable 
programs,  and  that  adjustments  to  ex¬ 
isting  programs  do  not  balance  the 
books  at  the  expense  of  our  strategic 
goals. 

To  make  sure  that  he  is  in  a  position 
to  do  this  effectively,  the  VCJCS  is 
directing  the  Joint  Staff  evolution  of  an 


improved  capability  to  conduct  cost- 
trade  analysis  in  support  of  develop¬ 
ing  fiscally  constrained  strategies. 

CINC  Spokesman  on  Acquisition 
And  Requirements 

The  VCJCS  will  be  sponsoring 
CINC  requirements  for  considera¬ 
tion/validation  when  normal  processes 
aren't  appropriate  or  aren't  equal  to  the 
task.  As  noted,  most  of  this  function 
will  be  assumed  by  the  JROC  but  the 
final  responsibility  will  rest  with  the 
Vice  Chairman. 

This  means  the  VCJCS  will 
direct/request  Joint  Staff/Service 
assistance  to  CINCs  in  developing  Mis¬ 
sion  Need  Statements  and  presenting 
Mission  Area  Analysis  concerns. 
Ultimately,  however,  the  VCJCS  must 
balance  CINC  theater  needs  againsf  the 
tendency  to  proliferate  theater  specific 
weapon  systems. 

Program  Manager 

The  OJCS  primary  concern  is  the 
growing  time  required  to  move  from 
promising  technologies  to  deployed 
systems.  From  our  perspective,  pro¬ 
gram  instability  is  key;  it  begins  when 
requirements  are  defined  poorly  or 
become  moving  targets  as  the  threat 
changes  over  long  development 
periods.  It  is  aggravated  by  a  decision 
process  which  allows  bureaucratic 
"gate-keepers"  to  delay  programs  for 


narrow  parochial  concerns.  Finally, 
our  budget  process  can  leave  even  our 
best  program  managers  focusing  their 
skills  on  reducing  budget  uncertainty 
through  program  advocacy  rather  than 
reducing  cost,  schedule,  and  perfor¬ 
mance  risk  through  sound  program 
management. 

The  Defense  Reorganization  Act  and 
NSDD  219  have  provided  the  OJCS 
with  the  means  to  help  reduce  these 
sources  of  instability  so  that  program 
managers  can  concentrate  on  the  inter¬ 
nal  elements  of  their  programs  a  little 
more  and  the  externals  a  little  less. 
During  the  coming  years,  program 
managers  can  expect  to  see  more  sister- 
Service  uniforms  in  the  meeting  room, 
more  joint  funding  initiatives,  and  less 
tolerance  for  the  discovery  of  "dis¬ 
jointed  efforts." 


Lieutenant  Colonel  Wain  is  a  Weapon 
Systems  Program  Evaluator  for  the  Joint 
Staff.  As  such,  he  prepares  Chairman  and 
Vice  Chairman,  Joint  ChteJ's  of  Staff,  to 
participate  in  SECDEF  Performance 
Reviews,  Defense  Resources  Board,  Defense 
Acquisition  Board,  and  Joint  Requirements 
Oversight  Council  activities  on  USA  F  ma¬ 
jor  system  programs.  Fie  is  the  Joint  Staff 
acquisition  policy  analyst. 


GANTT  MEDAL  IS  AWARDED 


William  S.  Lee  III  has  received  the 
Henry  Lawrence  Gantt  Medal  from  the 
American  Society  of  Mechanical 
Engineers  and  the  American  Manage¬ 
ment  Association.  The  medal  is 
awarded  annually  for  distinguished 
achievement  in  management  as  a  ser¬ 
vice  to  the  community.  It  was 
established  in  1929  and  memorializes 
Henry  Lawrence  Gantt,  engineering 
manager,  industrial  leader,  and 
humanitarian. 

Mr.  Lee,  chairman  and  chief  ex¬ 
ecutive  officer  of  Duke  Power  Com¬ 
pany,  headquartered  in  Charlotte, 
N.C.,  was  recognized  before  1,500  in 
Chicago  "for  the  forward  looking 
policies  that  have  made  his  organiza¬ 
tion  a  leader  in  providing  opportunities 
for  minorities;  for  outstanding  leader¬ 
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ship  examples  in  the  development  of 
safe,  dependable  power;  for  his  con¬ 
tinuing  efforts  to  protect  both  the  en¬ 
vironment  and  the  communities  in 
which  his  organization  operates;  for 
his  unflagging  efforts  to  aid  education 
through  organizing  scholarship  pro¬ 
grams  within  his  organization  and  sup¬ 
port  councils  to  help  meeting  the  needs 
of  public  schools,  as  well  as  his  own 
personal  efforts  to  aid  students;  for  his 
overall  management  skills  which  have 
made  his  organization  a  leader  in  the 
field." 

The  American  Society  of  Mechan¬ 
ical  Engineers  members  on  the  Gantt 
Medal  Board  include  David  D.  Acker 
of  DSMC,  Frank  S.  Locke,  Nathan  H. 
Hurt,  Jr.,  William  K.  McAleer,  and 
Robert  Simmons. 

5 


AFIT  SEEKING 
FACULTY 

The  Air  Force  Institute  of  Tech¬ 
nology  (AFIT)  teaches  systems  acquisi¬ 
tion/management  courses  to  Air  Force 
and  DOD  personnel  in  programs 
leading  to  masters  degrees  and  in¬ 
programs  of  continuing  professional 
education.  It  is  seeking  well-qualified 
individuals  interested  in  teaching  in 
those  programs.  Expressions  of  interest 
should  be  addressed  to:  AFIT/LS  (Dr. 
W.  Mauer),  Wright-Patterson  AF3, 
Ohio  45433-6583.  Questions  should  be 
addressed  to  Professor  William  Dean, 
(513)  255-3355  or  AV  785-3355. 
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PMC  88-1  GRADUATES 
HEAR  DR.  COSTELLO 


HI p 

111  he  Honorable  Robert  8. 

*  Costello,  Under  Secretary  of 
Defense  (Acquisition),  in  his  remarks 
to  the  Program  Management  Course 
88-1  graduating  class  at  the  Defense 
Systems  Management  College  in  May, 
said  "this  graduation  is  significant 
because  it  is  the  first  of  three  DSMC 
will  have  during  1988  under  the  new 
accelerated  PMC  structure,  replacing 
the  former  throughput  of  two  classes 
annually. 

"I  recognize  the  tremendous  effort 
all  of  you  on  the  faculty  and  staff  have 
put  into  making  this  new  process 
work.  I  know  the  overlap  in  classes 
tended  to  be  confusing  at  first,  but  it 
all  came  together. 

"Things  have  changed  at  DSMC, 
most  prominently  the  mission.  Now, 
DSMC  is  responsible  for  all  acquisition 
education,  rather  than  just  program 
management  education  as  before. . .  .To 
each  of  you  here  today,  continue  to 
work  not  only  for  the  good  of  the  col¬ 
lege,  but  for  improving  everything  we 
do  in  defense  acquisition." 

Dr.  Costello  said  graduates  are 
military  and  civilian  officials  and  are 
among  tomorrow's  top  government 
and  industry  leaders.  Each  has  been 
recognized  as  having  high  potential  to 
be  the  best.  "A  lot  is  riding  on  your 
shoulders,"  he  said,  "and  the  directions 
and  actions  our  country  takes  in  the 
future  will  depend  upon  judgments  and 
decisions  you  and  counterparts  will 
make.  It  is  an  awesome  responsibility 
and  must  be  handled  judiciously.  If 
there  is  one  single  point  I  wish  to  drive 
home  today,  it  is  that  Department  of 


Defense  people  must  make  changes  in 
mindset,  form  and  substance— the 
total  picture — in  order  to  improve  the 
way  we  do  business.  What  we  in  the 
Department  of  Defense  have  set  out  to 
change  will  be  done  gradually,  not 
overnight,  but  it  must  be  done. 

"Changes  must  be  initiated  at  the 
top,  encouraged  at  the  top,  im¬ 
plemented  at  the  top,  and  led  at  the 
top.  Top  management  must  be  com¬ 
mitted  to  change,  and  then  follow 
through  to  accomplishment. 

Meet  Challenges 

"Whether  you  will  be  in  manage¬ 
ment  or  staff  positions,  you  must  sup¬ 
port  efforts  of  those  above  you  to 
make  the  necessary  changes,  and  ful¬ 
ly  execute  them  down  the  line.  As  you 
advance  in  your  careers,  these  and 
other  challenges  await  you.  Meet  them 
head-on. 

"Do  not  be  satisfied  with,  or  fearful 
of,  established  institutions  as  mono¬ 
lithic  intimidators.  Believe  that  the 
Department  of  Defense  way  of  doing 
business  can  be  done  better.  Have 
courage  of  your  conviction  to  put  in¬ 
to  practice  only  that  which  is  best  for 
our  fighting  forces,  and  for  national 
security.  Discard  the  superfluous. 

Financial  Constrictions 

"Your  leadership  ability  will  be 
tested  severely  in  coming  years.  Tight 
financial  constrictions... will  be  with  us 
for  the  forseeable  future.... Many  of 
you  will  work  on  the  program  objec¬ 
tives  memorandum  and  budget  at  the 
Pentagon  and  at  headquarter  levels 


....You  will  wrestle  with  probable 
budget  cuts,  program  delays,  stretch¬ 
outs  and  perturbations.  Your  man¬ 
agerial  skill  to  get  the  most  for 
available  dollars  will  bear  on  your  job 
performance  but,  most  importantly, 
will  greatly  affect  the  end  product — 
whether  it  be  weapon  platforms, 
strategic  systems,  communications, 
size  of  forces,  or  numbers  of 
people. 

"The  management  and  leadership 
principles  you  have  honed  at  the 
Defense  Systems  Management  College 
will  prepare  you  well  for  years  to 
come,  and  help  you  make  it  through 
exciting,  turbulent  and,  occasionally, 
stressful  times." 

Dr.  Costello  said  that  it  stands  to 
reason  that  management  and  leader¬ 
ship  principles  improving  the  acquisi 
tion  process  apply  across-the-board,  in 
every  facet  of  government.  He  told 
graduates  they  will  inherit  many  deci¬ 
sions  of  the  past;  some  will  bind, 
others  will  not.  "Your  contribution  to 
government,  and  the  payoff  of  your 
DSMC  education  will  be  to  use  proper 
initiative,  common  sense,  and  judg¬ 
ment  to  make  the  ultimate  hard 
decisions — and  make  them  right,"  he 
said. 

Dr.  Costello  concluded  his  remarks 
by  saying  that  "it  takes  leadership  to 
convince  people  and  established  in¬ 
stitutional  processes  that  changes  will 
make  everyone's  job  easier,  and  pro¬ 
vide  the  best  and  least  expensive 
systems  for  the  field. 


These  are  excerpts  from  Dr.  Costello's 
graduation  address.  His  ten  agenda  items 
for  improving  defense  acquisition  were  de¬ 
tailed  in  the  May-June  1988  issue  of 
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Major  General  Lynn  H.  Stevens,  USA ,  tenth  Com¬ 
mandant  of  the  Defense  Systems  Management  College. 


M; 


I  \h  /  111  ajor  General  Lynn  H. 
JL  ▼  JIL  Stevens,  USA,  was  Depu¬ 
ty  Chief  of  Staff  for  Development, 
Engineering  and  Acquisition,  Army 
Materiel  Command,  before  becoming 
DSMC's  tenth  Commandant  April  29. 
He  has  held  important  command  and 
staft  positions  including  Director  oi 
Materiel  Plans  and  Programs  in  the 
Office  of  the  Deputy  Chief  of  Staff  for 
Research,  Development  and  Acquisi¬ 
tion,  U.S.  Army;  Project  Manager, 
HAWK  Missile  System,  Redstone 
Arsenal,  Ala.;  Project  Manager, 
ROLAND  II  Missile  Systems, 
Redstone  Arsenal;  and  Commanding 
General,  U.S.  Army  Materiel  Develop¬ 
ment  and  Readiness  Command, 
Europe. 

In  Vietnam,  General  Stevens  was 
Executive  Officer,  later  Materiel  Of¬ 
ficer,  191st  Ordnance  Battalion,  500th 
Transportation  Group,  and  Opera¬ 
tions  Officer  (S-3),  500th  Transporta¬ 
tion  Group.  U.S.  Army  Cam  Ranh 
Bay  Support  Command. 

Other  assignments  in  his  30-year 
military  career  include  Assistant  for 
SAFEGUARD  and  Selected  Acquisi¬ 
tion  Reports,  Office  of  the  Assistant 
Secretary  of  the  Army  (Installations 
and  Logistics);  Commander,  197th 
Ordnance  Battalion,  59th  Ordnance 
Group,  USA  Europe;  Chief,  Nuclear 
Weapons  Staff,  Office  of  the  Deputy 
Chief  of  Staff  for  Logistics,  USA 
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NEW  COMMANDANT 


The  "flag  is  passed “  at  the  Change  op' Command  Ceremony  April  29  at  the  Defense  Systems 
Management  College,  Ft.  Belvoir,  Va. 

Europe  and  Seventh  Army;  and 
Logistics  Staff  Officer  in  the 
Maintenance,  Policy,  and  Programs 
Division,  Supply  and  Maintenance 
Directorate,  Office  of  the  Deputy 

Chief  of  Staff  for  Logistics,  USA  ^ 

Washington. 

General  Stevens  returned  overseas  Jf 

as  Chief,  Munitions  and  Missiles  Divi- 
sion,  Office  of  the  Deputy  Chief  of  V:\; 

Staff  for  Logistics,  USA  Europe  and 
Seventh  Army,  and  later  became  Com-  ••.' 

mander,  59th  Ordnance  Brigade,  USA 
Europe. 


(leneral  Stnyns  (left)  cuts 
cake  with  help  of  The 
Honorable  Robert  R. 
Costello,  Under  Secretary  of 
Defense Jbr  Actjuisinon.  and 
Brigadier  Central  Charles 
P.  Cabell,  Jr.,  IISAF,  retir¬ 
ing  ( ammandant. 
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DISPELLING  MYTHS  OF 
TEST  AND  EVALUATION 


Dr.  H.  Steven  Kimmel 

Assistant  Deputy  Director 
Defense  Research  and  Engineering 
(Test  and  Evaluation) 


T: 


I  est  and  Evaluation  is 
-ML  recognized  as  a  key 
element  of  the  weapon 
system  process.  By  long¬ 
standing  practice  and  direc¬ 
tive,  weapon  system  test  and 
evaluation  is  divided  into 
two  principal  categories— 

Development  Test  and  Eval¬ 
uation  (DT&E)  and  Opera¬ 
tional  Test  and  Evaluation 
(OT&E).  As  defined  by  the 
governing  directive  (DoDD 
5000.3,  "Test  and  Evalua¬ 
tion"),  DT&E  is  conducted 
throughout  various  phases 

of  the  acquisition  process  to  ensure  the  acquisition  and 
fielding  of  an  effective  and  supportable  system  by  assisting 
in  the  engineering  design  and  development  and  verifying  at¬ 
tainment  of  technical  performance  specifications,  objectives 
and  supportability.  The  OT&E  is  the  field  test,  under 
realistic  conditions  and  by  typical  users  of  the  weapon 
system  (or  element  thereof)  to  determine  its  suitability  and 
effectiveness. 

While  DT&E  emphasizes  engineering  design  and  technical 
performance,  its  ultimate  goal,  like  that  of  OT&E,  is  to  en¬ 
sure  the  acquisition  and  fielding  of  weapon  systems  that  are 
effective  and  supportable  under  combat  conditions.  One 
should  not  exoect  DT&E,  by  itself,  to  be  sufficient  to  fully 
ensure  effective,  supportable  combat  operation.  Key 
elements  of  realistic  testing  are  reserved  to  OT&E;  e.g., 
operation  by  typical  military  users  in  as  realistic  a  represen¬ 
tative  field  condition  as  possible  against  threat  representative 
hostile  forces.  Nonetheless,  it  is  clear  that  the  utility  of  DT&E 
as  an  acquisition  tool  is  increased  when  Development  Test 
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(DT)  results  can  serve  as  a 
reliable  predictor  of  Opera¬ 
tional  Test  (OT)  perfor- 


jtr 


Figure  1  presents  a 
simplified  model  of  the 
weapon  system  acquisition 
process  to  assist  in  examin¬ 
ing  the  relationship  between 
DT&E  and  OT&E.  The 
model  suggests  that  test  and 
evaluation  is  a  continuum  of 
activities  interwoven  with 
the  acquisition  process.  In 
reality,  the  maturing  DT&E 
and  initial  phases  of  OT&E  (IOT&E)  events  do  not  fit  into 
rigid  or  discrete  compartments;  both  are  involved  with 
broad,  system-level  concerns.  This  relationship  is  a  matter 
of  ongoing  interest  and  often  the  cause  of  confusion  and 
misunderstanding.  In  addition,  recent  enactments  by  the 
Congress  have  drawn  attention  to  the  Office,  Secretary  of 
Defense  (OSD)  T&E  communities'  management,  execution 
and  actions.  The  result  of  this  certainly  needs  ti  be  well 
understood  by  those  engaged  in  the  acquisition  of  weapon 
systems.  Accordingly,  the  reader  is  invited  to  take  and 
evaluate  the  test  on  the  next  page. 

The  intent  of  this  test  is  not  to  provide  pass/fail  criteria 
for  the  reader,  but  to  clarify  ongoing  relationships  between 
DT&E  and  OT&E  communities.  More  precisely,  the  entire 
T&E  community  is  being  challenged  to  support  the  evolv¬ 
ing  acquisition  community  goals  and  objectives.  Using  viable 
test  programs  and  the  commitment  to  objective  assessments 
are  the  essential  means  to  achieve  results  recently  expressed 
by  the  Army  Chief  of  Staff  General  Carl  E.  Vuono: 
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Test  and  Evaluation  is  very 
important  because  putting  less 
than  adequate  weaponry  into  the 
hands  of  our  soldiers  is  a  price 
that  we  all  cannot  afford. 


Fifteen  Questions 

1.  The  OSD  Test  and  Evaluation  en¬ 
vironment  is  divided  and  fragmented. 

T  or  F 

2.  Test  and  Evaluation  assessments 

go  unheeded.  T  or  F 

3.  Weapon  system  acquisition  time 
has  increased  due  to  the  demands/ re¬ 
quirements  for  more  testing.  T  or  F 

4.  Existing  T&E  policies  of  DOD  and 
military  departments  are  inconsistent 
in  philosophy  and  approach.  T  or  F 

5.  Development  Test  (DT)  and  Initial 

Operation  Test  (IOT)  are  incompati¬ 
ble  in  purpose.  T  or  F 


6.  Test  and  Evaluation  Master  Plans 

(TEMPs)  are  needed  solely  to  comply 
with  DoD  Directive  5000.3,  "Test  and 
Evaluation";  i.e.,  to  fulfill  a  process  re¬ 
quirement.  T  or  F 

7.  Until  recently,  software  T&E  has 

placed  emphasis  more  on  "T"  than  "E" 
and  has  thus  endorsed  the  Design-Test- 
Redesign-Retest  philosophy  of 
development.  T  or  F 

8.  Live  Fire  Testing  is  solely  an  Army 
test  program  brought  about  by  the 
Bradley  Live  Fire  Test  (LFT)  effort. 

T  or  F 

9.  Non-development  item  (ND1)  is  an 

acquisition  strategy  that  eliminates  the 
requirement  for  Test  and  Evaluation 
master  planning.  T  or  F 

10.  A  balanced  structured  system  test 

plan  treats  hardware,  software  and  the 
user  as  equals.  T  or  F 

11.  Preplanned  product  improvements 

(P3I)  or  evolutionary  acquisition  (EA) 
strategies  (yes,  there  is  a  difference) 
minimize  the  conflict  between  DT  and 
OT.  T  or  F 
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12.  Concurrent  acquisition  strategies 
typically  reduce  schedule  risk  at  the  ex¬ 
pense  of  informed  decisions.  T  or  F 

13.  Operational  suitability  and  opera¬ 

tional  effectiveness  are  graduation 
characteristics  validated  as  a  conse¬ 
quence  of  OT.  T  or  F 

14.  Nuclear  hardness  and  survivabili¬ 
ty  (NH&S)  objectives  are  to  be 
achieved  in  DT  and  confirmed  in  OT. 

T  or  F 

15.  Weapon  system  (platform)  com¬ 
bat  survivability  is  a  function  of 
susceptibility  and  vulnerability.  T  or  F 


Answers  to  Questions  with  Sup¬ 
porting  Rationale 

1.  The  OSD  Test  and  Evaluation  en¬ 
vironment  is  divided  and  fragmented. 

False.  That  is,  while  the  OSD  T&E 
function  is  fragmented  organizational¬ 
ly,  its  purpose  is  not.  Clearly,  the 
Director  of  Operational  Test  and  Eval¬ 
uation  (DOT&E)  is  responsible  for 
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providing  OT  policy,  while  the  Depu¬ 
ty  Director  of  Defense  Research  and 
Engineering  (Test  and  Evaluation), 
DDDRE(T&E),  is  responsible  for  DT 
policy  generation.  However,  both  en¬ 
tities  agree  that: 

— "Test  planning"  must  begin  early 

—Early  test  results  are  essential  to  sup¬ 
port  design  to  production  decisions 

—The  DT  objectives  must  posture  a 
system  OT 

— The  OT  must  be  viewed  as  a  gradua¬ 
tion  type  of  event  rather  than  a  final 
exam. 

A  single  OSD  organization  would 
not  alter  this  ideology.  Two  organiza¬ 
tions  permit  focused  attention  upon  a 
myriad  of  related  matters  such  as  test 
facilities  (instrumentation  and  ranges), 
test  targets,  threat  surrogates  and 
simulators,  live  fire  testing,  joint  tests, 
and  foreign  weapon  evaluation.  Most 
importantly,  it  allows  independent, 
objective  weapon  systems  assessments 
from  a  development  perspective  to  en¬ 
sure  that  engineering  thresholds  have 
been  attained  and  from  an  operational 
perspective  to  verify  the  suitability  and 
effectiveness  is  confirmed  before  pro¬ 
ceeding  beyond  low-rate  initial 
production. 

2.  Test  and  Evaluation  assessments 
go  unheeded. 

F; '  The  T&E  assessments  are  an 
integral  influence  in  the  defense  ac¬ 
quisition  decision-making  process. 
Routine  vehicles  such  as  weekly  "quick 
look"  T&E  correspondence  and  more 
detailed  status  reports  and  memoran¬ 
da  are  frequently  generated  to  ensure 
the  Secretary  of  Defense  and  Under 
Secretary  of  Defense  (Acquisition)  are 
appraised  of  acquisition  T&E  matters. 
Lastly,  the  T&E  assessments  provided 
to  the  Defense  Acquisition  Board  often 
form  the  foundation  of  the  resultant 
Acquisition  Decision  Memorandum 
that  directs  the  Service  Acquisition  Ex¬ 
ecutive  during  the  forthcoming 
months  of  program  execution. 

3.  Weapon  system  acquisition  time 
has  increased  due  to  the  demands/re¬ 
quirements  for  more  testing. 

False.  During  1987,  as  a  result  of 
congressional  interest,  a  thorough  and 
comprehensive  review  of  the  weapon 

Program  Manager 


system  acquisition  process  judged  that 
a  vigorous,  well-planned  T&E  effort  is 
time-efficient  and  a  wise  investment. 
The  conclusion  from  the  evaluation  of 
testing  time  is  that  T&E  is  not  a 
reasonable  target  for  time  reduction  in 
pursuit  of  a  speedier  acquisition  pro¬ 
cess,  and  that  a  cutback  in  T&E  could 
actually  lead  to  a  longer  and  costlier 
acquisition  process. 

4.  Existing  T&E  policies  of  DOD  and 
military  departments  are  inconsistent 
in  philosophy  and  approach. 

False.  The  DcDD  5000.3,  "Test 
and  Evaluation,"  sets  forth  the  broad 
philosphical  basis  for  T&E  and  iden¬ 
tifies  specific  responsibilitites  and 
methods  for  all  T&E  actions  in  the 
Department  of  Defense.  It  highlights 
the  purpose  of  testing,  the  relationship 
between  DT  &  OT  as  currently  de¬ 
fined,  specific  military  service  respon¬ 
sibilities,  and  T&E  planning  and  ex¬ 
ecution  requirements. 

The  military  services,  in  turn,  have 
each  prepared  regulations  that  imple¬ 
ment  the  guidance  contained  in  DoDD 
5000.3  for  their  respective  organiza¬ 
tions.  In  general,  the  military  services 
are  in  concert  with  the  fundamentals 
of  the  DOD  T&E  guidance,  although 
there  are  differences  in  terminology 
and  approach.  For  the  most  part,  this 
is  a  consequence  of  the  different 
weapon  system  classes  and  operating 
environments  with  which  each  mili¬ 
tary  service  must  deal. 

5.  Development  Test  (DT)  and  Initial 
Operational  Test  (IOT)  are  incompati¬ 
ble  in  purpose. 

False.  The  T&E  is  an  integral  part 
of  the  acquisition  process,  interacting 
with  other  program  functions  that  sup¬ 
port  the  development  of  a  product 
design  and  helping  to  determine  the 
operational  effectiveness  and  suitabili¬ 
ty  of  a  weapon  system.  Testing  (devel¬ 
opment  and  operational)  must  begin 
early  and  be  done  continuously  rather 
than  viewing  it  as  a  "final  exam.”  Ac¬ 
cordingly,  the  relationship  between 
DT  and  IOT  is  separate,  yet  com¬ 
plementary.  For  example,  an  early 
operational  assessment  is  often  based 
upon  the  DT  data.  In  the  end,  they 
contribute  to  a  healthy  T&E  environ¬ 
ment  which,  in  turn,  helps  the  DOD 
in  its  decision  process.  The  DT  con¬ 


tributes  to  the  design  portion  of  the 
process,  whereas  OT  establishes  the  in¬ 
formation  base  for  recommending 
when  a  program  is  ready  to  proceed 
past  a  low-rate  initial  production. 

6.  Test  and  Evaluation  Master  Plans 
(TEMPs)  are  needed  solely  to  comply 
with  DoDD  5000.3,  'Test  and  Evalua¬ 
tion";  i.e.,  to  fulfill  a  process 
requirement. 

False.  The  purpose  of  a  TEMP  is 
to: 

—Identify  the  scope  of  planned  testing 
—Delineate  acceptable  evaluation 
criteria 

— Foster  sound  program  management 
by  which  to  execute  a  rational,  logical 
course  of  action. 

The  concept  of  a  TEMP  is  de¬ 
tailed  in  DoD  Manual  5000.3-M-l, 
Test  and  Evaluation  Master  Plan 
Guidelines,  and  supports  the  premise 
that  the  document  should  be  viewed  as 
a  tool  to  ensure  program  management 
success.  The  TEMP  should  serve  as  a 
roadmap  rather  than  a  detailed  test 
plan.  Accordingly,  program  officials 
should  use  it  as  a  vehicle  to  confirm, 
via  test  results,  the  progress  of  a 
maturing  set  of  events.  This  progress 
is  examined  continuously  during  the 
development  period  and  is  necessary 
to  facilitate  the  certification  that  the 
developing  system  under  test  is  ready 
for  OT.  Utilizing  evolutionary 
thresholds  that  span  the  phases  of  ad¬ 
vanced  and  engineering  development 
right  through  the  certification  for  OT 
readiness  and  leading  to  initial  opera¬ 
tional  capability  can  become  bona  fide 
checkpoints  to  support  acquisition 
management  decisions. 

Hence,  the  TEMP  is  and  should  be 
viewed  as  a  living  document,  accom¬ 
modating  annual  updates  and  revi¬ 
sions.  Meanwhile,  the  TEMP  must 
track  with  the  approved  requirements 
and  Decision  Coordination/System 
Papers  to  ensure  that  the  objective 
system,  once  fully  fielded,  satisfies  the 
military  user's  required  operational 
needs  (final  set  of  evolutionary 
thresholds). 

Accordingly,  military  service  sub¬ 
mitted  TEMPs  receive  a  thorough  and 
comprehensive  OSD-wide  review  to 
ensure  that  programmatic  and  tech- 
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nical  viewpoints,  concerns  and  details 
are  adequately  addressed. 


7.  Until  recently,  software  T&E  has 
placed  its  emphasis  more  on  "T"  than 
"E"  and  has  endorsed  the  Design-Test- 
Redesign-Retest  philosophy  of 
development. 


True.  The  design-test-redesign  ap¬ 
proach  has  been  observed  to  be  costly 
(see  note  below)  and  inefficient  for 
hardware  and  software  development 
efforts.  Therefore,  the  soon-to-be 
published  DoD  Manual  5000.3-M-3 
"Software  Test  and  Evaluation 
Manual"  will  advocate  the  following: 


1.  DoD-STD-2167,  "Defense 
Systems  Software  Development," 
which  established  a  tri-service  ap¬ 
proach  for  designing  and  building  in 
software  quality  vice  testing  in  soft¬ 
ware  quality.  Quality  is  one  measure 
of  software  maturity. 


2.  Development  of  mathematical 
means  of  determining  software  correct¬ 
ness  analytically  during  design  and 
prior  to  code  development,  leading  the 
way  to  error  reduced  software  being 
developed  in  a  "clean-room"  environ¬ 
ment.  This  is  still  in  its  infancy. 


3.  Support  the  DOD  implemen¬ 
tation  of  Ada  for  use  in  all  weapon 
systems.  This  approach  fosters  the  ear¬ 
ly  detection  and  prevention  of  require¬ 
ment  and  design  errors  prior  to  soft¬ 
ware  coding. 


4.  Improving  management  visi¬ 
bility,  testing,  and  assessment  tools 
which  promote  management  attention, 
early  detection,  and  correction  of  soft¬ 
ware  problem  areas. 

NOTE:  Evidence  produced  by  soft¬ 
ware  studies  indicates  that,  on  the 
average,  approximately  40-60  percent 
of  DOD  software  development  dollars 
are  spent  on  software  test-related 
activities. 


8.  Live  Fire  Testing  is  solely  an  Army 
test  program  brought  about  by  the 
Bradley  Live  Fire  Test  (LFT)  effort. 


False.  Vulnerability  of  the  Bradley 
Fighting  Vehicle  was  initially  quan¬ 
tified  by  the  OSD  sponsored,  joint 
Live  Fire  Test  effort.  In  FY  87,  the 
Congress  established  a  DOD-wide 
Live  Fire  Test  mandate  to  be  overseen 
by  OSD  and  executed  by  each  military 
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service.  The  enabling  legislation  states 
that  LFT  shall  be  executed  sufficiently 
early  in  the  development  phase  of  the 
system  or  program  to  allow  any  design 
deficiency  demonstrated  by  the  vulner¬ 
ability  testing  to  be  corrected  in  the 
design  of  the  system,  munition  or 
missile  before  proceeding  beyond  low- 
rate  initial  production. 


False.  The  use  of  NDI  is  based 
upon  the  results  of  a  market  surveil¬ 
lance  and  analysis  performed  early  in 
the  system  life  cycle.  Typically,  the 
analysis  determines  the  feasibility  of 
satisfying  a  military  deficiency  or  need 
by  utilizing  commercial  off-the-shelf 
products.  These  products  may  be  used 
either  directly,  or  ruggedized  for 
military  environments,  or  integrated 
into  existing  or  evolving  system 
design(s).  In  any  case,  the  degree  of 
testing  will  be  commensurate  with  the 
degree  of  integration  and/or  modifica¬ 
tion  required.  It  will  be  inversely  pro¬ 
portional  to  the  depth  of  contractor 
data  appropriate  for  evaluating  mili¬ 
tary  operational  suitability  and 
effectiveness. 


As  described  in  the  forthcoming 
DoD  5000.3-M-5,  "Procedures  Man¬ 
ual-Improving  Test  and  Evaluation  Ef¬ 
fectiveness  in  Support  of  the  Major 
Weapon  Systems  Decision  Process," 
the  testing  of  an  NDI  must  not  impede 
the  objectives  of  streamlining  the  ac¬ 
quisition  process.  Rather,  it  must  be 
accomplished  in  an  orderly,  objective 
fashion  consistent  with  the  approved 
TEMP. 


10.  A  balanced  structured  system  test 
plan  treats  hardware,  software  and  the 
user  as  equals. 


The  TEMP  is  the  umbrella  document 
to  record  the  scope  of  a  system,  muni¬ 
tion  or  missile's  LFT.  The  supporting 
Detailed  Live  Fire  Test  and  Evaluation 
Plan  contains  the  subsystem  and  com¬ 
ponent  or  full-up  testing  needed  to 
assess  system  vulnerability  or  lethali¬ 
ty.  Such  a  full-up  test  may  be  waived 
by  the  Secretary  of  Defense  prior  to 
entering  Full-Scale  Development  and 
provided  that  the  Secretary  certifies  to 
the  Congress  that  LFT  would  be  un¬ 
reasonably  expensive  and  impractical . 


True.  A  balanced  test  plan  is 
necessary  to  ensure  that  system  and 
mission  objectives  will  be  supported  by 
maturing  hardware  and  software.  The 
test  plan  must  address  hardware  (in¬ 
cluding  mission  critical  computer 
resources  as  well  as  front-end  sensors), 
software  (firmware  as  well  as  resident 
and  data  flow  induced),  user  elements 
and  the  integration  of  all  of  these 
elements.  System-level  testing  should 
be  designed  and  conducted  to  demon¬ 
strate  the  contribution  of  hardware, 
software  and  people  to  the  quantifica¬ 
tion  of  reliability,  availability,  and 
maintainability  parameters. 


9.  Non-development  item  (NDI)  is  an 
acquisition  strategy  that  eliminates  the 
requirement  for  Test  and  Evaluation 
master  planning. 


11.  Preplanned  product  improvements 
(P3I)  or  evolutionary  acquisition  (EA) 
strategies  (yes,  there  is  a  difference) 
minimize  the  conflict  between  DT  and 
OT. 


True.  The  P3I  and  EA  contain  a 
modular  building  block  concept  to 
enable  the  integration  of  progressive 
hardware  and  software  design  en¬ 
hancements  capable  of  meeting 
futuristic  mission  objectives  by  the 
phasing-in  of  upgradable  intrinsic 
elements. 


The  enhancements  result  in  cost 
avoidance  in  such  areas  as  obsolete 
system  software,  planned  delivery  of 
new  technology  and  the  tailoring  of 
operational  characteristics  to  increase 
the  utility  of  available,  employable 
technology. 


Hence,  at  the  system  level,  the  adap¬ 
tation  of  P3I  or  EA  tends  to  minimize 
DT  and  OT  conflicts  by  accommo¬ 
dating  design  changes  during  a  spec¬ 
trum  of  time  as  evidenced  by  specific 
development  objectives  and  OT  expec¬ 
tations.  With  a  balanced  test  program 
consisting  of  integration,  interopera¬ 
bility,  and  compatibility  testing,  the 
modular  concept  can  provide  the  road¬ 
map  to  reduce  test  and  acquisition 
conflicts. 


12.  Concurrent  acquisition  strategies 
typically  reduce  schedule  risk  at  the  ex¬ 
pense  of  informed  decisions. 


False.  In  the  usual  context,  concur¬ 
rency  means  either  tke  simultaneous 
DT /OT  or  the  more  common  interp re- 
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tation  of  simultaneous  DT/OT  and 
production. 

With  concurrent  DT/OT  the  pro¬ 
gram  office  and  contractor  are  faced 
with  the  dilemma  of  trying  to  collect 
developmental  data  as  quickly  as 
possible  while  providing  and  support¬ 
ing  a  system  for  the  independent 
operational  tester  to  verify  user  re¬ 
quirements.  What  often  happens  is  the 
curtailment  or  retarding  of  develop¬ 
mental  data  taking  to  ensure  com¬ 
pliance  with  a  particular  operational 
performance  threshold,  thus  impeding 
overall  development  and  progress 
toward  certification  for  dedicated 
system  level  OT.  Some  concurrency  in 
DT /OT  is  beneficial  but  only  to  the  ex¬ 
tent  that  quick-look  OT  evaluations 
are  used  to  assist  the  developer  and 
help  refine  system  specifications. 

D'  -rimental  concurrency  of  DT/OT 
occurs,  for  example,  when  the  pro¬ 
gram  office/contractor  team  candidly 
admits  their  attention  has  been 
diverted  from  developmental  to  opera¬ 
tional  testing.  Typically,  this  means 
that  the  preference  to  collect  data  (i.e., 
to  fire  several  telemetry  equipped 
missiles)  followed  by  the  opportunity 
to  correct  engineering  deficiencies  has 
been  set  aside  in  deference  to  the 
necessity  to  rehearse  IOT&E  to 
minimize  the  possibility  of  any  embar¬ 
rassing  operational  occurrences.  Thus, 
the  attainment  and  confirmation  of 
development  objectives  as  verified 
through  flight  test  become  mitigated. 

13.  Operational  suitability  and  opera¬ 
tional  effectiveness  are  graduation 
characteristics  validated  as  a  conse¬ 
quence  of  OT. 

True.  The  latter  phase  of  OT  (i.e., 
OTII),  is  the  place  to  prove  operational 
effectiveness.  Whereas  some  of  opera¬ 
tional  suitability  can  be  assessed  from 
the  results  of  DT,  DoD  5000.3  states 
that  operational  suitability  under 
realistic  conditions  is  to  be  validated 
during  OT. 
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14.  Nuclear  hardness  and  survivabili¬ 
ty  (NH&S)  objectives  are  to  be 
achieved  in  DT  and  confirmed  in  OT. 

True.  The  DoDI  4245.4,  "Acquisi¬ 
tion  of  Nuclear-Survivable  Systems," 
states  that  "NH&S  objectives  are  (to 
be)  achieved  during  DT  and  OT&E." 

In  addition,  DoDD  5000.3,  "Test 
and  Evaluation,"  states  that  "DOT&E 
is  responsible  for  (confirming),  in  coor¬ 
dination  with  the  Assistant  to  the 
Secretary  of  Defense  for  Atomic 
Energy,  ATSD(AE),  that  OT&E  con¬ 
firms  NH&S  as  intended." 

"DDDRE  (T&E)  is  responsible  for 
confirming,  with  advice  from  the 
ATSD(AE),  that  NH&S  objectives  are 
achieved  during  DT&E." 

As  we  are  aware,  testing  to  confirm 
NH&S  must  depend  upon  a  combina¬ 
tion  of  tests,  simulations  and  analysis. 
Like  the  non-nuclear  survivability 
area,  severity  of  the  mission  degrada¬ 
tion  will  be  extrapolated  in  terms  of  the 
ability  of  the  threat  operating  environ¬ 
ment  to  exceed  the  system  design  capa¬ 
bilities,  thereby  affecting  system  sur¬ 
vivability.  The  degree  to  which  the 
system  survives  will  directly  contribute 
to  mission  abort,  mission  degradation, 
nuisance  or  no  response.  In  the  end, 
the  system's  design  (development  ob¬ 
jective)  must  be  robust  to  lie  some¬ 
where  between  impervious  to  the  ex¬ 
pected  threat  (i.e.,  no  vulnerability), 
to  built-in  recovery  (observable  degra¬ 
dation)  and  the  acceptance  of  partial 
(but  militarily  acceptable)  operational 
suitability  and  effectiveness. 

15.  Weapon  system  (platform)  com¬ 
bat  survivability  is  a  function  of 
susceptibility  and  vulnerability. 

True.  Survivability  is  "the 
capability  of  a  system  to  avoid  and/or 
withstand  a  man-made  hostile  environ¬ 
ment"  without  suffering  abortive  im¬ 
pairment  of  its  ability  to  accomplish  its 
designated  mission. 


One  element  of  survivability  must 
be  expressed  in  terms  of  the  intrinsic 
system  ability  to  use  whatever  possi¬ 
ble  natural  phenomena,  countermeas¬ 
ures  or  tactics  to  avoid  weapons 
(ballistic  projectiles,  guided  missiles, 
exploding  warheads,  and  other  ele¬ 
ments  that  make  up  the  hostile  en¬ 
vironment).  That  is,  the  measure  can 
be  expressed  by  Ph,  the  probability 
that  the  system  may  be  hit  by  a 
damage-causing  mechanism,  and  is 
referred  to  as  the  susceptibility  of  the 
system. 

The  inability  of  a  system  to  with¬ 
stand  the  damage  caused  by  the  hostile 
environment  is  referred  to  as  the 
vulnerability  of  the  system  to  the 
damage  mechanisms.  Vulnerability 
can  be  measured  by  P,/h,  the  condi¬ 
tional  probability  that  tne  system  will 
be  killed  or  degraded  given  a  hit  by  a 
damage  mechanism. 

The  extent  that  a  system's  effec¬ 
tiveness  may  be  reduced  by  the  hostile 
environment  is  measured  by  the  prob¬ 
ability  the  system  is  killed  Pk  or 
degraded  Pd.  The  probability  of  kill  of 
the  system  is  the  product  of  the  prob¬ 
ability  of  hit  (the  susceptibility),  P. , 
and  the  conditional  probability  of  kill 
given  a  hit  (the  vulnerability),  Pk/h, 
that  is: 

Probability  of  Kill  _  Susceptibility 
*  Vulnerability  Pk  =  Ph  *  Pk/h  is  ap¬ 
propriate  for  conditions  of  permanent 
damage;  i.e.,  high  explosive  induced, 
and  Probability  of  Degradation  = 
Susceptibility  *  Vulnerability  Pd  =  Ph 
*Pd/h  is  appropriate  for  conditions  of 
temporary  damage;  i.e.,  electro¬ 
magnetic  induced. 

Thus,  the  survivability  of  the  system 
can  be  expressed  as  a  capability  to  en¬ 
dure  in  a  hostile  environment  as 
measured  by  the  probability  of  sur¬ 
vival  Ps.  Its  relationship  to  Pk  is  given 
by  P  =  1  -  Pk. 
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C/SCSC  LESSONS  LEARNED 
THEORETICAL  FRAMEWORK 

Dr.  Anthony  Webster 
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rogram  managers 
JL.  for  the  government, 
and  defense  contractors  have 
come  a  long  way  since  early 
implementation  of  the 
Arthur  D.  Little  Cost/ 

Schedule  Control  System 
Criteria  (C/SCSC)  in  the 
early  1967-68  time  frame. 

Currently,  there  is  less  ap¬ 
prehension  of  the  criteria, 
and  the  cost  to  implement 
appears  to  be  reasonable  in 
most  cases.  Contractor  and 
government  personnel  con¬ 
tinue  to  learn  more  and  bet¬ 
ter  ways  to  use  C/SCSC  and  analyze  performance  measure¬ 
ment  data.  Many  observations  have  been  made  and  many 
lessons  have  been  learned. 

In  1975,  a  C/SCSC  Implementation— Lessons  Learned- 
study  was  conducted  by  L.  S.  Morrella  and  J.  B.  Holeman 
as  a  project  during  the  Defense  Systems  Management  Col¬ 
lege  PMC  75-1.  That  paper  documented  the  "corporate 
memory"  of  C/SCSC  experiences  and  lessons  learned  for 
the  7-year  span,  1968-75. 

My  paper  builds  on  earlier  research,  attemping  to  com¬ 
pare  deficiencies  and  lessons  learned  identified  in  1975  with 
current  observations  of  deficiencies  in  the  discipline  (12  years 
later). 

Lessons  learned  as  observed  in  the  1975  study  are  pro¬ 
vided  in  Table  1. 
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Other  areas  of  co:  cem 
were  program  instability, 
schedule  variance  expressed 
in  dollars,  accounting  for 
material  at  point  of  usage, 
level  of  detail  involved,  cost 
associated  with  implement¬ 
ing  and  inflexible  interpreta¬ 
tion  of  criteria. 

The  Lessons  Learned  (Fig¬ 
ure  3,  pp.  20-22)  reflects  per¬ 
spectives  taken  from  seven 
vantage  points  as  a/an: 

—Contractor  performance 

measurement  instructor  working  with  user  population 
—Observer  of  feedback  from  C/SCSC  validation  reports 
— Participant  in  C/SCSC  focal  points  and  review  director 
meetings 

—Participant  in  NSIA  Management  Subcommittee  meetings 
—Consultant  providing  C/SCSC  support  to  government 
agencies 

—Member  of  the  Performance  Management  Association 
—Participating  team  member  on  demonstration  and  subse¬ 
quent  application  reviews. 

Background 

Observation  #1.  A  recent  Air  Force  study  reveals  that  of 
86  reviews  conducted  from  1984-87,  70.5  percent  were 
satisfactory,  11.4  percent  marginal  and  18.2  percent  un¬ 
satisfactory.  The  deficiency  distribution  by  criteria  categories 
for  those  reviews  are  shown  in  Table  2. 
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TABLE  1.  LESSONS  LEARNED  IN  1975  STUDY 

c/scsc 

SUBJECT 

REMEDY 

Retrofit  applications 
(not  cost  effective,  disruptive) 

Retrofit  applications  should  be 
avoided. 

Premature  demonstration 
reviews  (not  cost  effective, 
disruptive) 

Realistic  schedules  must  be 
developed.  Meaningful  readiness 
assessments.  Contractor  should 
not  drag  out  process. 

Lack  of  program  management 
office  involvement 

Encourage  Department  of  Defense 
(DOD)  program  managers  to  have 
office  well  represented  on  the  1 
review  team.  (Big  dividends  in 
terms  of  understanding  and 
visibility). 

Lack  of  headquarters 
guidance 

Provide  adequate  and  standard 
guidance  to  all  review  team 
members.  Make  it  the  responsi¬ 
bility  of  the  review  director  to  ! 

implement  a  standard  interpreta¬ 
tion  of  the  criteria  and  be  well- 
trained  in  the  C/SCSC  discipline. 

The  review  director  should  be  pro¬ 
vided  by  the  major  military  service 
headquarters. 

Lack  of  plant  representative 
involvement. 

Enforce  DOD  requirements  that  con¬ 
tract  administrators  and  auditors 
conduct  continuing  monitoring  and 
surveillance  of  contracts.  The  Cost 
Accounting  Standards  (CAS)  and 
Defense  Contract  Auditing  Agency 
(DCAA)  should  be  involved  in 
demonstration  reviews.  The 
Memorandums  of  Agreement 
(MOAs)  should  be  established. 

Top  management  support 

“Units  do  well  those  things  the  Boss 
checks.”  Top  management  on  i 
both  sides  must  embrace  and  sup¬ 
port  the  requirement. 

Analysis  and  use  of  data 

Help  contractor  and  government  \ 
analysts  to  have  a  better  apprecia¬ 
tion  of  data  value  and  emphasize 
a  greater  understanding  of  data 
analysis,  and  utilization. 

Estimates  at  Completion  (EACs) 
review  team  members 

Emphasize  the  need  for  meaningful 
and  disciplined  EACs.  The  EACs 
should  be  realistic  and  timely. 
Turnover  must  be  minimized  and 
adequately  trained. 
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TABLE  2.  CRITERIA  CATEGORIES 


NUMBER 


C/SCSC  CATEGORY 


PERCENT 


Organization 
Planning  and  budgeting 
Accounting 
Analysis 


Revisions  and  access  to  data 


TABLE  3.  DEFICIENCIES 


DEFICIENCY 


FREQUENCY  OF 
OCCURRENCE  PERCENT 


Estimate  of  cost  at  completion 
Selection  of  earned  value  technique 
Level  of  effort  application 
Subcontractor  performance  measurement 
Baseline  establishment  and  control 
Variance  analysis  and  reporting 
System  discipline 
Material  control 
Validity  of  computer  runs 
Reconciliation  of  external  reporting  data 
with  internal  data 


TABLE  4.  PROBLEMS  IMPLEMENTING  C/SCSC 


PROBLEM  CATEGORY 


REASONS 


System  problem 


Implementation  problem 


Government  problem 


System  not  required  for  years  (not 
updated) 

Deviations  from  validated  system 

Poor  internal  audits 

Training  schedule  traceability 

Application  of  earned  value 
techniques 

Quality  of  variance  analysis 

Undefinitized  contracts 

Unrealistic  target  cost 

Poor  surveillance 

Program  office  allows 
non-compliance 
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Other  areas  of  concern  by  the  Air 
Force  were  excessive  data  reporting  re¬ 
quirements,  delayed  reporting  of 
material  and  subcontractor  cost 
variances  and  qualifications  of 
C/SCSC  Review  Team  members  and 
Teeth  in  C/SCSC  (remedies)  for 
non-compliance . 


Observation  #2.  The  Navy  analyzed 
18  Validation  Review  Reports  of  con¬ 
tracts  totaling  $1.8  billion  and  iden¬ 
tified  the  10  most  frequently  reported 
deficiencies  that  could  affect  validity 
of  cost  data.  Table  3  shows  the  defi¬ 
ciencies  and  their  frequency  of  occur¬ 
rences  observed  during  the  18  reviews. 


Observation  §3.  In  a  recent  presen¬ 
tation  made  by  the  Tri-Service  Focal 
points,  current  problems  implementing 
C/SCSC  were  identified  and  divided 
into  three  major  categories— systems 
problems,  implementation  problems 
and  government  problems.  Table  4 
shows  the  information  provided  in  that 
presentation. 


Observation  #4.  In  an  interview 
with  a  C/SCSC  management  consul¬ 
tant  who  has  supported  the  govern¬ 
ment,  I  asked  him  to  provide  a  list  of 
major  deficiencies  in  implementing 
C/SCSC  based  on  his  company’s  ex¬ 
periences.  Data  provided  are  shown  in 
Table  5. 
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Observation  #5.  Feedback  from 
Contractor  Performance  Measurement 
Course  students,  C/SCSC  Seminars  I 
have  attended  for  the  last  5  years  and 
speaking  with  major  defense  contrac¬ 
tors  provide  a  composite  overview  of 
perceived  key  problems  in  the  im¬ 
plementation  of  C/SCSC. 

They  are: 

—Interfacing  with  existing  systems 

— Establishing  and  maintaining  WBS 
vs.  cost  accounts 

—Developing  code  fields  to  integrate 
cost /schedules 

—Establishing  objective  eamed-value 
techniques 

— Keeping  the  baseline  up-to-date 

—  Reporting  against  unrealistic 
baselines 

—Accounting  for  materials  and  multi¬ 
function  equipment 

— Handling  subcontractors  (especially 
in  research  and  development) 

—Using  Management  Reserve  as  a 
management  tool. 


TABLE  5.  MAJOR  DEFICIENCIES  IN 
IMPLEMENTING  C/SCSC 

PROBLEM  CATEGORY 

REASONS 

Planning 

—Incorrect  identification  of  cost 
accounts 

—Weak  relationship  between  the 

Work  Breakdown  Structure  (WBS) 
and  statement  of  work 
—Inadequate  development  of  the 

WBS 

—Poor  definition  of  work 
—Inadequate  integration  of  budget, 
schedule,  and  work  authorization 
—Overallocation  of  budget 
—Forward  planning  at  too  high  a 
level 

Control 

—Work  status  not  based  on 
work  package  progress 
—The  Budgeted  Cost  for  Work 

Performed  (BCWP)  technique  not  con¬ 
sistent  with  resource  application 

—Performance  measurement  at 
incorrect  level 

—Improper  or  late  incorporation  of 
changes 

—Invalid  baseline  maintenance 
system 

—Material  costs  not  applied  properly 

Accounting  system 

—Accounting  periods  do  not  match 
customer  or  subcontractor  account¬ 
ing  periods 

—The  12  vs.  13  accounting  periods 
during  the  year 

—Reports  prepared  after  accounting 
month  closing 

—Inadequate  follow-up  to  reduce 
errors 

—No  recording  of  labor  and  material 
transfers  between  departments  and 
projects 

Data  analysis 

—Variance  analysis  system  not 
maintained 

—Performance  inadequately 
monitored 

—Corrective  actions  not  followed  up 

—Estimates  at  completion  not 
updated 

System  maintenance 

—System  design  too  complex 
—Documentation  not  accurate 
—Training  not  done  continuously 

Program  Manager 
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TABLE  6.  OPPORTUNITIES  FOR  IMPROVEMENT 

PROBLEM  CATEGORY 

REASONS 

Baseline  change 

The  DOD  change  order  pro¬ 
cess  does  not  permit 
work  to  start  prior  to 
receipt  of  authorization. 

Lack  of  timely  accounta¬ 
bility  in  Performance 

Measurement  Baseline 
(PMB). 

Review  process 

Lack  of  consistency  in  the 
criteria  interpretation  be¬ 
tween  review  teams. 

Preconceived  changes 
directed  prior  to  the 
scheduled  review. 

C/SCSC  application 

Lack  of  flexibility  in  its 
implementation  during 
the  research  and  develop¬ 
ment  phase. 

Variance  analysis 

Formal  documentation  of 
analysis  is  time  con¬ 
suming. 

Some  reporting  is  repetitious,  extra 
paperwork. 

Thresholds  are  too  low. 

CPR  reporting 

The  levels  for  reporting 
requirements  are  too  low. 

WBS 

The  MIL-STD  881 -A  should  be 
a  guide,  not  a  mandate. 

TABLE  7.  MAJOR  AREAS  FOR  IMPROVEMENT 

PROBLEM  CATEGORY 

REASONS 

C/SCSC  review  process 

Team  member  qualifications  (not 
well  qualified). 

Turnover  rate  is  high. 

Career  path  problems. 

C/SCSC  application 

Lack  of  consistency  in  criteria. 

interpretation 

Time  allowed  for  implementation. 

Lack  of  incentives  for  a  contractor 
to  work  C/SCSC. 

Lack  or  understanding  of  C/SCSC 
capabilities  and  limitations  by 
both  government  and  industry. 

Cost  performance  reports 

Excess  reporting. 

Timeliness  of  reports. 

Quality  of  variance  analysis. 

Observation  § 6 .  Table  6  identifies 
areas  some  Defense  contractors  believe 
presented  the  greatest  opportunity  for 
improvement  in  the  implementation  of 
C/SCSC. 


Observation  #7.  The  C/SCSC 
survey  of  Department  of  Defense  and 
contractor  managers  Phase  II  was  also 
a  source  considered  for  background  in¬ 
formation.  Phase  II  of  referenced 
survey  identified  problems,  whose 
resolution  could  lead  to  improvements 
in  the  implementation  of  C/SCSC. 
The  major  areas  for  improvement 
identified  in  this  survey  are  provided 
in  Table  7. 


APPROACH 

Analysis  of  Data 

The  observations  give  background 
information  to  show  a  representation 
of  major  deficiencies/problems  asso¬ 
ciated  with  implementing  C/SCSC  and 
Lessons  Learned.  They  represent  a 
cross-section  of  government  and  in¬ 
dustry  program  management  sources 
throughout  the  country.  This  data 
covers  from  1968  through  1987.  The 
referenced  data  set  includes:  The 
C/SCSC  Study,  "Lessons  Learned," 
1968-75;  Air  Force  C/SCSC  Review 
Results,  1987;  Navy  C/SCSC  Review, 
1986;  C/SCSC  Tri-Service  Presenta¬ 
tion,  1987;  C/SCSC  Management 
Consultant  Interview,  1987;  Feedback 
from  CPMC  student  population  and 
C/SCSC  seminars,  1983-87;  Consor¬ 
tium  of  Defense  Contractors:  C/SCSC 
Response,  1987;  and  Arthur  D.  Little 
C/SCSC  Survey,  1986. 

The  deficiencies  identified  through 
the  data  set  were  stratified  into  27 
categories  and  put  into  a  matrix.  See 
Figure  1,  C/SCSC  Implementation 
Deficiency  Matrix,  to  determine  the 
frequency  of  occurrence  among  the 
eight  sources  used  in  the  data  set.  The 
frequency  distribution  is  shown  in 
Figure  2. 
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FIGURE  1. 

C/SCSC  DEFICIENCY 
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etrofit  C/SCSC  appl. 


Premature  demonstrations 


Lack  of  PO  involvement 


Lack  of  HQ  guidance 


Plant  representatives 


Top  management  support 


Analysis/use  of  data 


Estimate  at  completion 


Review  team  members 


Planning  and  budgeting 


Accounting 


Analysis  (variance) 


Earned  value 


Subcontractor 


Material  control 


Reports 


Training 


Schedules 


Authorized  undefinitized 


WBS 


Baseline 


Management  reserve 


LOE  application 


System  discipline 


Revisions 


Work  authorization 


Non-compliance 
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FIGURE  2.  THE  C/SCSC  DEFICIENCY  FREQUENCY  DISTRIBUTION  TABLE 


DEFICIENCY 

CATEGORY 

ORDER  OF  HIGHEST 
FREQUENCY  '(See  Note) 

BASIS  FOR  LESSONS 
LEARNED  TABLE 

1 .  Analysis/use  of  data 

6 

VARIANCE  ANALYSIS 

2.  Analysis  (variance) 

6 

3.  Training 

6 

TRAINING 

4.  Reports 

5 

REPORTS 

5.  Earned  value 

4 

EARNED  VALUE 

6.  Material  control 

4 

MATERIAL  CONTROL 

7.  Review  team  members 

4 

REVIEW  TEAM 

8.  Estimate  at  completion 

3 

EAC 

9.  Accounting 

3 

10.  Subcontractor 

3 

SUBCONTRACTOR 

11.  Baseline  maintenance 

3 

BASELINE 

12.  Work  breakdown  structure 

3 

WBS 

13.  Systems  discipline 

3 

SYSTEM  DISCIPLINE 

14.  Revisions 

3 

REVISIONS 

15.  Lack  of  program  office 

involvement 

16.  Top  management  support 
I  17.  Plant  representative 

18.  Planning  and  budgeting 

19.  Schedules 

20.  Authorized  undefinitized 

21.  C/SCSC  non-compliance 

22.  Lack  of  headquarters  involvement 

23.  Retrofit  C/SCSC  application 

24.  Premature  demonstrations 

25.  Management  reserve 

26.  LOE  application 

27.  Work  authorization 


TOP  MGMT  SUPPORT 


SCHEDULES 


MANAGEMENT  RESERVE 


'NOTE:  Frequency  of  occurrence  is  on  attached  C/SCSC  Implementation 
Deficiency  Matrix. 
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FIGURE  3.  THE  C/SCSC  LESSONS  LEARNED  (1987) 


SUBJECT 

PROBLEM 

SOLUTION 

VARIANCE  ANALYSIS 

.  Inherent  reaction  time  of  variance  data 
not  sufficient  to  predict  each  individual 
problem. 

.  Continue  to  stress  observation 
of  early  warning. 

.  Documenting  superficial  detail  problems 
can  be  time-consuming  and  non-productive. 

.  Set  variance  thresholds  to  maximize  ad¬ 
verse  trend  analysis. 

.  Lack  of  timely  and  meaningful  data/variance 
analysis. 

.  Stress  not  only  variance  thresholds. 

.  Quantify  the  variance. 

.  Describe  the  problem  and  its  impact  on 
cost,  schedule  and  other  elements. 

.  Recommendation/corrective  actions. 

.  Emphasize  program  manager  involvement. 

.  Too  stringent  threshold  requirements 
(Retard*;  proper  management  and  analysis). 

.  Thresholds  should  vary  depending  upon  size 
and  sensitivity  of  reporting  element. 

.  Value  of  Cost  Performance  Report  (CPR) 
schedule  variance  may  not  indicate  a  real 
schedule  problem. 

.  Don't  ignore  the  schedule  variance;  it  is  a 
preliminary  indicator  of  cost  problems  and  a 
useful  trending  tool. 

.  Variance  expressed  in  dollars... conversion  to 
time  can  be  a  useful  tool. 

TRAINING  AND  IMPLE¬ 
MENTATION  OF  C/SCSC 

.  Lack  of  consistency  in  the  interpretation 
of  the  criteria. 

.  Provide  team  members  with  work  instruc¬ 
tions;  flexible,  functional  procedures. 

.  Lack  of  training. 

.  Require  formal  training  for  team  members. 

.  Smal1  classes  and  on-the-job  training. 

.  Require  subteam  establishment  with  at 
least  one  qualified  member. 

.  Lack  of  C/SCSC  emphasis  at  top  levels  of 
management. 

.  Develop  and  distribute  a  C/SCSC 
newsletter. 

.  Lack  of  understanding  of  C/SCSC 
capabilities  and  limitations.  (Government 
and  Industry) 

.  Special  courses  for  commanders,  chief 
executive  officers  (CEOs)  and  vice 
presidents. 

.  Involve  top-level  management  in  the 
reviews. 

.  Lack  of  consistency  in  C/SCSC  application. 

.  Annual  meetings  of  C/SCSC  review  direc¬ 
tors  and  team  chiefs. 

.  Involve  the  participation  of  the  Office  of  the 
Secretary  of  Defense,  training  institutions, 
and  cross  fertilization  of  the  military 
services  on  review  teams. 

.  Preconceived  changes  directed  prior  to  the 
scheduled  review. 

.  Educate  the  contractor  about  the  appeals 
process. 

VARIANCE  ANALYSIS 

.  Analysis  and  use  of  the  data. 

,  Improvise  ways  to  help  contractor  and 
government  analyst  to  better  appreciate  the 
value  of  the  data  and  emphasize  a  greater 
understanding  of  how  to  analyze  and  utilize 
the  data. 

REPORTS 

.  Excessive  data/reporting  requirements. 

.  Validate  proposed  requirements  and  review 
current  requirements. 

.  Educate  personnel  on  optimal  data  needs. 

.  Delayed  reporting  of  materials  and  sub¬ 
contract  cost  variances. 

,  Require  tracking  in  Format  5  of  performance 
against  a  material  cost  baseline  as  material 
commitments  and  subcontract  awards  are 
made. 

FIGURE  3.  THE  C/SCSC  LESSONS  LEARNED  (1987)  (Continued) 

SUBJECT 

PROBLEM 

SOLUTION 

REPORTS  (cont’d.) 

.  Require  monthly  adjustments  of  EAC  based 
on  reported  material  cost  variances  and  ac¬ 
tual  commitment. 

.  Lack  of  reconciliation  of  external  reports 
with  the  CPR. 

.  In  Format  1,  require  reporting  by  cost 
element  (labor,  material  and  other  direct). 

.  Reconcile  CPR  with  Contractor  Cost  Data 
Report  (CCDR).  (Aids  in  cost  rejections.) 

.  Timeliness  of  the  reports,  and  quality  of. 
the  variance  analysis. 

.  Sen/ice  focal  points  should  assure  that  the 
contractor  is  aware  of  the  need  for  timely 
and  quality  data  for  the  reports  to  yield  their 
maximum  utility.  Use  the  contractual  vehicle 
to  assure  compliance. 

.  Incremental  submittal  of  CPR  Formats. 

.  Implement  the  use  of  electronic  data 
transmission  when  possible. 

EARNED  VALUE 

.  Insufficient  measurement  techniques. 

(Too  much  level  of  effort  (LOE)) 

.  Avoid  use  of  LOE. 

.  Use  all  available  quantitative,  objective 
measurement  techniques. 

.  It  is  better  to  revert  to  subjective  assess¬ 
ment  techniques  rather  than  allow  higher 
percentage  of  LOE. 

.  Maximize  ratio  of  measurable  vs.  LOE  to 
prevent  distortion  of  performance 
measurement. 

.  Mix  of  LOE  within  discrete  cost  accounts. 

.  Mixture  is  allowable  within  certain 
guidelines  and  constraints. 

MATERIAL  CONTROL 

.  Lack  of  material  accountability  and  status 
on  contracts. 

.  Ensure  that  the  system  provide  visibility 
into  the  entire  material  status  of  contract. 

a.  Should  be  identified  in  WBS  element. 

b.  People  responsible  for  material. 

c.  Earned  value  for  material. 

ESTIMATE  AT  COMPLETION 
(EAC) 

.  Adequacy  of  EAC  projection  difficult  to 
assess:  Frequency,  application  of  automa¬ 
tion.  application  of  unreliable  overhead 
rates. 

.  Evaluate  EACs  by  trend  analysis. 

.  Require  periodic  detail  EAC  preparation. 

.  Intensify  use  of  “Cum-to-date”  cost 
variances. 

.  Cost  growth  tends  to  occur  in  steps. 
Reluctance  of  manager  to  admit  growth. 
Time  constant  between  problem  identity 
and  cost  impact. 

.  Supplement  manager  estimates  with 
parametric  techniques. 

.  Allow  for  program  uncertainties  when 
establishing  management  reserve  (MR). 

SUBCONTRACTOR 

.  Lack  of  subcontractor  visibility. 

(Sometimes  the  subcontractor  does  not  ap¬ 
pear  in  the  prime  contract  work  breakdown 
structure  (CWBS). 

.  Management  Control  System  flowdown 
required. 

.  Must  be  addressed  in  solicitation  and  con¬ 
tracts  clause. 

.  Prime  tasked  to  perform  validation  and 
surveillance  to  subcontractor. 

.  Subcontractor  reporting  non-standard. 

.  Should  be  compatible  with  the  prime,  via  CPR, 
C/SSR  with  required  procedures  for  taking 
BCWP. 

.  Subcontractor  EACs. 

.  EACs  must  be  monitored  for  the 
subcontractor. 
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FIGURE  3.  THE  C/SCSC  LESSONS  LEARNED  (1987)  (Continued) 


SUBJECT 

PROBLEM 

SOLUTION 

BASELINE 

.  Baseline  establishment  and  control. 

Internal  trace  of  work  from  contract  line  item 

(Weak  relationship  between  WBS  and  state- 

numbers  (CLINs)  to  SOW,  to  WBS  cells/WBS 

ment  of  work  (SOW). 

dictionary. 

.  Baseline  not  representative  of  all  work  on 

Baseline  should  reflect  status  of  authorized 

the  contract. 

unpriced  work. 

Subcontractors  must  appear  in  the  baseline. 

SYSTEM  DISCIPLINE 

.  Manufacturing  cost  accounts 

.  Establish  cost  accounts  at  meaningful  cost 

—Inappropriate  WBS  levels. 

account  levels. 

—Arbitrary  time  spans. 

.  Allov,  cost  accounts  to  represent  total  task. 

.  Multiple  budgeting  and  tracking  techniques. 

Adapt  system  to  internal  methods. 

.  Government/contractor  system  under- 

Establish  working  rapport  with  government 

standing. 

surveillance  monitors.  ! 

.  System  not  in  use  for  years. 

Establish  internal  focal  points. 

Periodic  self-audits. 

Use  system  to  manage  all  programs. 

.  Authorizations 

All  authorizations  which  include  tasK,  budget. 

—Approval  signatures  dated  after  work 
starts,  or  no  dates. 

and  schedule. 

—Many  log  entries  dated  the  week  before 

Maintain  current  logs  for  changes,  MR  and 

the  arrival  of  the  review  team. 

undistributed  budget. 

REVISIONS 

.  Authorized  undefinitized  work. 

.  Obtain  Procuring  Contracting  Officer  (PCO) 

(Not  incorporating  effort  into  the  PMB  can 

Authorization  to  proceed  and  incorporate  j 

jeopardize  contract  performance.) 

proposal/NTE  value  as  an  authorized 
undefinitized  change. 

.  Timeliness  of  internal  changes. 

.  Changes  require  immediate  incorporation 
into  the  baseline. 

SCHEDULES 

.  Lack  of  schedule  traceability  (horizontal) 

.  Contractual  requirement  for  submission  of 

detailing  interdependencies: 

network  schedules. 

—Problem  (intermediate  level  schedules) 

.  Expansion  of  intermediate  schedules  to 

no  horizontal  trace  between  key  events 

identify  interdependencies. 

and  major  milestones. 

.  Require  quarterly  submission  of  summary 

.  Lack  of  sufficient  predecessor-successor 
identification. 

network. 

MANAGEMENT  RESERVE 

.  Reserve  source  must  not  be  speculative 

.  Segregate  reserves  for  labor  rate  fluctuation 

(e  g.,  labor  rate  windfalls). 

from  technical  or  program  contingencies. 

.  Criteria  for  application  must  be  definitive  and 

.  Reserves  should  be  controlled  by  higher 

restrictive. 

management. 

Conclusion 


This  article  tells  the  C/SCSC  story 
and  provides  corporate  memory  of 
C/SCSC  problems/solutions  from 
1968-87.  The  data  were  gathered  from 
a  cross-section  of  government  and  in¬ 
dustry  program  management  sources 
throughout  the  country. 

Overall,  this  data  reflects  a  history 
of  steady  progress  toward  successful 
implementation  of  the  discipline. 
Much  of  the  data  indicates  there  has 
been  a  transition  from  problems  of  im¬ 
plementation  to  analysis  and  use  of 
data.  Training  and  analysis  and  use  of 
data  stood  out  as  priority  among  the 
problems  currently  associated  with  im¬ 


plementing  C/SCSC.  The  data  also  in¬ 
dicate  that  the  discipline  still  lacks  the 
kind  of  top  management  support  that 
is  conducive  to  future  success  in  im¬ 
plementing  C/SCSC.  This  suggests 
that  government  and  industry  top 
management  should  be  more  involved 
in  the  review  process  and  perhaps  a 
top-level  management  C/SCSC  course 
should  be  established. 

Nonetheless,  in  collecting  this  data 
it  became  obvious  there  was  a  lack  of 
consistency  in  how  C/SCSC  data  has 
been  collected  over  the  years.  It  has 
ranged  from  that  of  subjective  surveys 
to  frequency  of  deficiency  occurrences. 


Based  on  this  observation,  I  recom¬ 
mend  establishing  a  set  of  standard 
parameters  for  data  collection  for  Air 
Force,  Army,  Navy,  NSIA,  and  other 
C/SCSC  data  sources.  In  time,  this 
collection  of  a  standard  data  set  will 
provide  a  greater  opportunity  to 
analyze,  compare  and  extract  pertinent 
information  from  our  C/SCSC  data 
bases. 


Dr.  Webster  is  a  Professor  of  financial 
management  at  the  Defense  Systems 
Management  College. 
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111  his  fall,  a  document  will  be  issued  describing  the  best 
-JL-  practices  in  quality  and  productivity  measurement 
and  management  on  defense  systems.  Those  of  us  concerned 
with  preparing  this  document  hope  it  will  gain  widespread 
acceptance  in  the  defense  and  aerospace  industry  because 
current  practices  need  to  be,  and  can  be,  improved. 

The  importance  of  measurement  was  recognized  by  Lord 
Kelvin  in  1883  when  he  stated:  "I  often  say  that  when  you 
can  measure  what  you  are  speaking  about  and  express  it  in 
numbers,  you  know  something  about  it;  but  when  you  can¬ 
not  express  it  in  numbers,  your  knowledge  is  of  a  meagre 
and  unsatisfactory  kind;  it  may  be  the  beginning  of 
knowledge,  but  you  have  scarcely,  in  your  thoughts, 
advanced  to  the  stage  of  science,  whatever  the  matter  may 
be."1 

We  should  not  deal  with  quality  and  productivity  as 
though  they  are  intangibles.  Rather,  we  should  deal  with 
them  in  numbers;  otherwise,  results  will  be  of  "a  meagre 
and  unsatisfactory  kind." 

The  "Creed  for  Productivity  Improvement,"  as  stated  by 
J.  L.  Riggs  and  G.  H.  Felix  in  their  textbook  on  productiv¬ 
ity,  underlines  an  important  point  1  would  like  to  make.2 
According  to  these  writers. 

To  improve  productivity  you  must  manage. 

To  manage  effectively,  you  must  control. 

To  control  consistently,  you  must  teasure. 

To  measure  validly,  you  must  define. 

To  define  precisely,  you  must  quantify. 


^ j  Quality  and  productivity  measurement  and  management, 

g  once  considered  to  be  arduous  tasks  have  been  improved, 
J*  but  can  be  improved  further.  Recognizing  this,  some 
knowledgeable  people  working  in  university-based  produc¬ 
tivity  centers  and  leading  defense  industrial  firms  in  the 
United  States  have  been  engaged  in  a  five-year,  five-phased, 
management-oriented  quality  and  productivity  project, 
funded  by  the  Department  of  Defense,  to  see  how  and  where 

(improvements  might  be  made.  Findings  have  been  reviewed 
by  an  industry  advisory  group,3  and  the  Office  of  Secretary 
of  Defense  and  military  service  representatives  at  selected 
checkpoints  throughout  the  project.  Results  will  be  published 
in  the  fall. 

I  participated  in  the  project  from  the  beginning  and  served 
as  the  contracting  officer's  representative  for  three  of  the 
five  years.  This  landmark  project  addressed  important  issues 
in  quality  and  productivity  measurement  and  management 
I  on  defense  programs.  Before  publication  the  guide,  setting 
[  forth  project  results,  is  being  reviewed  and  evaluated  by 
I  executives  and  managers  of  industrial  firms,  and  the  Office 
I  of  the  Secretary  of  Defense  (OSD)  and  Military  Department 
I  representatives.  The  guide  will  incorporate  recommended 
i  changes. 

r  Let's  review  project  phases  leading  to  preparation  and 
publication. 
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Summary  of  Efforts  By  Project 
Phase 

Phase  I  was  conducted  under  leader¬ 
ship  of  Monte  Norton  and  Wayne 
Zabel  at  the  Army  Procurement 
Research  Office  (APRO),  Fort  Lee, 
Va.,  with  encouragement  from 
Dr.  Richard  A.  Stimson,  then  Direc¬ 
tor  of  Industrial  Productivity,  Office 
of  the  Under  Secretary  of  Defense 
(Research  and  Engineering).4  The 
effort  involved  a  survey  of  then- 
current  productivity  measurement  and 
evaluation  techniques  used  in  the 
defense  industry.  Some  definitions 
relative  to  contractor  productivity, 
acceptable  to  the  defense  industry, 
were  developed. 

The  Phase  I  study  revealed  several 
things. 

—As  a  performance  evaluation  factor, 
industrial  contractors  ranked  produc¬ 
tivity  below  profitability,  effec¬ 
tiveness,  quality,  and  efficiency. 

—  Problems  encountered  in  industrial 
firms'  attempts  to  measure  productiv¬ 
ity  are  usually  caused  by  difficulties 
experienced  in  identifying  and  quan¬ 
tifying  inputs  and  outputs. 
—Visibility  of  production  costs  varies 
widely  in  industry.  In  addition, 
visibility  of  indirect  costs  is  becoming 
more  important  because  of  shifts  in 
cost  drivers. 

—Tracking  the  impact  of  improve¬ 
ment  interventions  is  difficult,  espe¬ 
cially  in  indirect  cost  areas. 
—Investments  are  made  more  often  for 
competitive  and  technological  reasons 
than  for  reducing  costs  on  current 
contracts. 

— Industry  must  learn  to  integrate  con¬ 
ventional  and  new  approaches  to  per¬ 
formance  measurement  and  evalua¬ 
tion,  applying  them  when  and  where 
appropriate.5 
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TABLE  1.  MODELS  AND  METHODOLOGIES  FOR 
MEASURING  PRODUCTIVITY 


MODE  U 

METHODOLOGY 


DESCRIPTION 


Total  Factor  The  TFPMM  Is  a  dynamic,  aggregated,  indexed,  and 

Productivity  Measurement  computerized  approach  to  measuring  productivity. 


Model  (TFPMM) 


Cost  Definition 

Methodology 

(CDEF) 


TFPMM  can  be  utilized  to  measure  productivity 
changes  in  labor,  capital,  materials,  energy  and 
data/information.  It  measures  the  corresponding  effect 
each  one  has  on  profitability. 

The  TFPMM  is  based  on  the  premise  that  profitability 
is  a  function  of  productivity  and  price  recovery;  that  is, 
an  organizational  system  can  generate  profit  growth 
from  productivity  improvement  and/or  from  price 
recovery.  Productivity  relates  to  quantities  of  output 
and  quantities  of  inputs,  while  price  recovery  relates 
to  prices  of  output  and  costs  of  inputs. 


The  Price  Waterhouse  CDEF  methodology  is  an 
approach  for  preparing  cost  baseline  data  in  support 
of  factory  modernization  efforts  for  both  commercial 
factories  and  factories  participating  in  the  Department 
of  Defense  Industrial  Modernization  Incentives  Pro¬ 
gram  (IMIP).  CDEF  utilizes  a  top-down  analysis  tech¬ 
nique  which  facilitates  the  identification  of  appropriate 
performance  and  cost  measurement  criteria,  selection 
of  improvement  opportunities  (through  capital  invest¬ 
ment  and/or  efficiency  improvements),  and  economic 
justification  of  identified  investments. 


Discounted  Cash  Flow/  The  DCF/SSA  model  consists  of  (1)  a  time-phased 
Shared  Savings  Approach  model  of  cash  receipts  (or  savings)  and  cash 
(DCF/SSA)  disbursements  over  a  finite  planning  horizon,  with  a 

portion  of  the  receipts  being  shared  savings,  and  (2)  a 
calculated  internal  rate  of  return  measure  of  economic 
effectiveness. 


LTV  Integrated  The  LTV  model  incorporates  elements  of  the  TFPMM, 

Productivity  Measurement  CDEF,  and  DCF/SSA  methodologies  for  measuring 
Model  productivity. 


Phase  II  was  conducted  by  the 
Oklahoma  Productivity  Center  at 
Oklahoma  State  University,  and  the 
Maryland  Center  for  Productivity  and 
Quality  of  Working  Life  at  the  Univer¬ 
sity  of  Maryland,  under  contract  to  the 
Air  Force  Business  Research  Manage¬ 
ment  Center  (AFBRMC).  Colonel 
Ronald  Deep,  USAF,  Director  of 
AFBRMC,  was  the  contracting  officer's 
representative.  This  phase  focused  on 
identifying  and  describing  available 


productivity  measurement  techniques 
and  developing  a  taxonomy  of  produc¬ 
tivity  measurement  theories  and 
techniques. 

Phase  II  concluded  three  things. 

—Knowledge  of  specific  productivity 
measurement  techniques  is  not 
widespread. 

—State-of-the-art  techniques  require 
substantial  efforts  to  implement. 

— Some  macro-measurement  and  sur- 
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rogate  techniques  may  be  adequate  for 
a  specific  manager's  needs.6 

Phase  III  was  conducted  under  con¬ 
tract  to  the  Defense  Systems  Manage¬ 
ment  College  (DSMC)  by  the  Virginia 
Productivity  Center  (VPC)  at  Virginia 
Polytechnic  Institute  and  State  Univer¬ 
sity.  Dr.  D.  Scott  Sink,  Director  of 
VPC,  was  principal  investigator.  This 
phase  focused  on  evaluating  produc¬ 
tivity  measurement  techniques  iden¬ 
tified  in  Phase  II;  i.e..  Total  Factor  Pro¬ 
ductivity  Measurement  Model 
(TFPMM),  Price  Waterhouse  Cost 
Definition  Methodology  (CDEF),  Dis¬ 
counted  Cash  Flow/Shared  Savings 
Approach  (DCF/SSA),  and  LTV  Inte¬ 
grated  Productivity  Measurement 
Model.  Models  and  methodologies  for 
measuring  productivity  are  presented 
in  Table  1.  Four  subcontractors  plus 
principals  directly  involved  in  Phase  III 
were  the  Maryland  Center  for  Produc¬ 
tivity  and  Quality  of  Working  Life, 
Dr.  Thomas  C.  Tuttle;  LTV  Aircraft 
Products  Group,  Shoni  Dhir;  Price 
Waterhouse,  Betty  Thayer  and 
William  Muir;  Westinghouse  Defense 
Group,  Richard  Engwall. 

Phase  III  concluded  several 
things. 

—Individually,  none  of  the  first  three 
models  evaluated  could  accomplish  all 
objectives  desired  by  industrial  firms 
or  government. 

— A  methodology  incorporating  a 
variety  of  measurement  and  evaluation 
models  is  required.  Models,  when  inte¬ 
grated  (as  was  done  by  LTV),  con¬ 
stitute  a  potentially  satisfactory 
methodology  to  accomplish  industry 
and  government  objectives;  that  is, 
industrial  firms  want  to  improve  com¬ 
petitiveness  and  profits.  On  the  other 
hand,  the  government  wants  to  lower 
acquisition  costs  and  improve  product 
quality. 

— Variances  in  organizations,  manage¬ 
ment  styles,  pressures,  priorities, 
perceived  problems  and  opportunities, 
and  skilled /competent  management 
personnel  make  it  difficult  to  translate 
and  transfer  models  and  methodolo¬ 
gies  from  one  company  to 
another.7 

Phase  IV  was  conduced  by  the 
Virginia  Productivity  Center  and  the 
subcontractors  identified  previously. 


under  contract  to  the  Defense  Systems 
Management  College  with  funding 
provided  by  the  Office  of  the  Secretary 
of  Defense.  This  phase  consisted  of  a 
field  test  of  the  TFPMM,  CDEF,  and 
DCF/SSA  and  the  LTV  Integrated  Pro¬ 
ductivity  Measurement  Models;  com¬ 
pletion  of  the  productivity  manage¬ 
ment  methodology  designed  in  Phase 
III ;  design  and  development  of  a  draft 
guide  to  communicate  philosophies, 
principles,  tools,  and  techniques  of 
quality  and  productivity  management 
(generally  referred  to  as  performance 
management);  and  development  of 
detailed  plans  for  evaluation 
workshops  to  be  held  in  Phase 
V.8  9 

Phase  V  is  conducted  by  the  Virginia 
Productivity  Center  and  subcontrac¬ 
tors  identified  previously  under  con¬ 
tract  to  the  Defense  Systems  Manage¬ 
ment  College  with  Office  of  the 
Secretary  of  Defense  funding.  The  goal 
is  to  complete  and  publish  the  guide 
drafted  in  Phase  IV.  Feedback  from  an 
evaluation  workshop  during  this  phase 
should  enhance  the  quality  of  the  guide 
before  release. 

The  Guide 

The  guide  will  describe  good  perfor¬ 
mance  measurement  and  management 
practices  used  in  the  defense  industry 
in  America  today.  Dr.  Sink  and  Paul 
Rossler,  the  project  manager,  and 
members  of  the  project  team  are  con¬ 
vinced  the  measurement  and  manage¬ 
ment  practices  described  in  the  new 
guide  will  help  each  organization 
achieve  and  maintain  a  competitive 
position  in  the  defense  system  acquisi¬ 
tion  business  in  the  1990s  and  beyond. 

The  performance  management  pro¬ 
cess,  which  is  easy  to  conceptualize 
(see  Figure  1)  is  not  easy  to  place  in 
operation.  If  a  manager  understands 
the  basic  process,  but  fails  to  obtain  a 
proper  understanding  ar.d  agreement 
within  the  organization  on  the  process 
and  specific  tools  needed  to  effectively 
implement  it,  that  organization  will 
not  reach  its  goal. 

The  process  shown  in  Figure  1  may 
appear  complex  at  first  glance; 
however,  it  is  a  simple  depiction  of 
how  many  excellently  run  companies 
in  America  measure  and  manage  qual- 


The  Guide 

will  describe 
good  performance 
measurement  and 
management 
practices  used  in  the 
defense  industry  in 
America  today. 


ity  and  pi  Juctivity.  It  depicts  the 
organization  as  an  open  system  with 
various  feedback  loops  from  the  envi¬ 
ronment,  and  it  highlights  interrela¬ 
tionships  among  components  of  a  per¬ 
formance  management  process. 

Probably  the  best  place  to  start 
describing  the  model  is  in  the  middle 
with  the  box  titled  "Organizational 
System.''  This  box  represents  the 
system  being  managed.  It  could  be  a 
company,  division,  plant,  or  depart¬ 
ment.  The  system  has  upstream 
systems  (internal  and  external  suppliers 
and  vendors)  which  provide  inputs  in 
the  forms  of  labor,  material,  capital, 
energy,  and  data /information.  The 
system  takes  these  inputs  and  converts 
them  into  outputs  in  the  form  of  goods 
and  services.  Downstream  systems 
(internal  and  external  customers)  react 
to  these  outputs,  creating  outcomes 
(customer  satisfaction,  readiness, 
profitability). 
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At  the  top  of  Figure  1  is  the  new 
competition  the  organization  must 
respond  to  in  order  to  compete  in  a 
global  economy.  This  new  competition 
and  global  economy  influence  com¬ 
pany  business  strategy  and  company 
visions  of  the  organization  of  the 
future.  The  company  assesses  its  pres¬ 
ent  performance,  and  uses  this  data  as 
a  foundation  for  developing  plans  for 
performance  improvement.  Key  per¬ 
formance  indicators  are  identified  to 
provide  the  company  with  feedback  on 
its  progress. 

Out  of  the  performance  improve¬ 
ment  planning  process  comes  specific 
performance  improvement  interven¬ 
tions,  tactics  and  techniques.  Note  that 
these  interventions  happen  at  five 
quality  checkpoints:  (Ql)  upstream 
systems;  (Q2)  inputs;  (Q3)  process; 
(Q4)  outputs;  and  (Q5)  downstream 
systems.  After  interventions  to  the 
system  have  been  made,  performance 
is  measured,  assessed,  and  analyzed  at 
the  five  quality  checkpoints  to  deter¬ 
mine  whether  the  impact  the  interven¬ 
tions  were  to  have  made  on  the  system 
actually  occurred.  Based  on  this  data, 
an  evaluation  relative  to  the  com¬ 
pany’s  business  strategy,  environment 
(internal  and  external),  vision,  plans, 
and  improvement  interventions  are 
made.  Note  that  the  process  of  evalua¬ 
tion  is  separate  from  the  process  of 
measurement.  Measurement  supports 
improvement  and  this,  of  course,  is  of 
primary  interest. 

If  a  company  has  an  effective,  high- 
quality  management  process  in  the 
areas  of  planning,  measuring  and 
evaluating,  and  controlling  and 
improving,  it  will  achieve  its  vision  of 
the  future,  and  its  desired  outcomes. 

Before  I  comment  on  the  guide  in 
more  detail,  it  is  important  that  you 
have  a  basic  understanding  of  the 
quality  and  productivity  terminology 
used. 
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FIGURE  1.  THE  PERFORMANCE 
MANAGEMENT  PROCESS 


Performance  Improvement  Planning  and 
Measurement  Systems  Development 


Performance  Improvement  Interventions: 
Tactics  and  Techniques 
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Organization  of  the  Future: 
—Growth 

—Competitiveness 
—Desired  Outcomes 


Definitions  of  Basic  Terms  Used  in  The  Guide 


The  definitions  listed  below 
apply. 

Effectiveness.  The  degree  to  which 
a  system  accomplishes  what  it  sets  out 
to  accomplish.10 

Efficiency.  A  comparison  among  re¬ 
sources  expected  to  be  consumed  to 
accomplish  a  specific  goal,  task,  or 
objective  and  resources  actually  con¬ 
sumed  to  reach  that  goal,  accomplish 
the  task,  or  meet  the  objective.11 

Innovation.  Creative  process  of 
adapting  products,  services,  and  pro¬ 
cesses  to  changing  internal  and  exter¬ 
nal  pressures,  demands,  changes,  and 
needs.  It  might  be  called  the  process  of 
maintaining  fitness  for  use  from  the 
customer's  viewpoint. 


Performance  (Quality  and  Produc¬ 
tivity)  Management.  Process  which  in¬ 
cludes  planning,  measurement  and 
evaluation,  control,  and  improvement. 
Figure  1  depicts  the  general  flow  of  the 
performance  management  process. 

Productivity.  Relationship  of  the 
quantity  of  output  goods  and/or  ser¬ 
vices  obtained  from  an  organizational 
system  for  some  period  of  time  to  the 
quantity  of  input  resources  consumed 
by  that  organizational  system  for  the 
same  period  of  time,  quality 
considered. 

Productivity  Improvement.  Result 
of  managing  and  intervening  in  key 
transformation  processes.  Productivity 
improvement  occurs  when  any  of  the 
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following  conditions  are  made  to 
exist: 

—Output  increases  and  input 
decreases 

— Output  increases  and  input  remains 
constant 

—Output  increases  and  input  increases 
but  at  a  lower  rate 
— Output  remains  constant  and  input 
decreases 

—Output  decreases  and  input 
decreases  but  at  a  more  rapid  rate. 

Productivity  Measurement.12  Selec¬ 
tion  of  physical,  temporal,  and/or 
perceptual  measures  for  input  variables 
and  output  variables,  and  the  develop¬ 
ment  of  a  ratio  of  output  measure(s) 
to  input  measure(s).  A  good  produc¬ 
tivity  measurement  system  will  in¬ 
dicate  to  management  when  there  is  a 
need  to  plan  and  act,  and  provide  clues 
where  in  the  system  to  intervene. 

Profitability.  Ability  to  yield  advan¬ 
tageous  results.  It  is  a  measure,  or  set 
of  measures,  of  the  relationship  among 
financial  resources  and  their  uses,  such 
as  return  on  assets  and  return  on 
investments. 


Quality.  Degree  to  which  a  system 
conforms  to  requirements,  specifica¬ 
tions,  and  expectations.  To  be  man¬ 
aged  effectively,  quality  must  be  de¬ 
fined  relative  to  five  quality  check¬ 
points,  namely: 

Ql:  Selection  and  management  of 
upstream  systems  (internal  and  exter¬ 
nal  suppliers  and  vendors) 

Q2:  In-coming  quality  control 

Q3:  In-process  quality  management 

Q4:  Out-going  quality  control 

Q5:  Proactive  and  reactive 
assurance  of  customer  satisfaction. 

These  checkpoints  were  highlighted  in 
Figure  1. 

Quality  of  Work  Life.  Efforts  to  in¬ 
crease  performance  through  improve¬ 
ment  in  psychosocial  rather  than 
technical  and  structural  systems. 

Total  Quality  Management.  Man¬ 
agement  system  including  all  five 
quality  checkpoints.  (See  "quality" 
definition.) 


Performance  of  an  organization  is  a 
function  of  many  of  the  criteria 
already  defined.  Conceptually,  these 
criteria  are  interrelated  in  the  manner 
shown  in  Figure  2. 

Desired  Outcomes  for  the  Guide 

Those  playing  parts  in  developing 
the  guide  hope  it  will  do  the 
following: 

— Communicate  fundamentals  of  per¬ 
formance  management  methodology 
—Establish  role  and  importance  of 
productivity  measurement  and 
management 

— Move  people  in  government  and  in¬ 
dustry  away  from  a  narrow  view  of 
productivity  measurement  and 
management 

— Demonstrate  that  measurement 
leads  to  improvement 
— Demonstrate  that  measurement  sup¬ 
ports  incentive  methodology. 


FIGURE  2.  THE  CASUAL  RELATIONSHIPS  AMONG 
PERFORMANCE  CRITERIA 
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Concept  of  Guide 

The  guide  will  set  forth  state-of-the- 
art  practices  based  upon  the 
knowledge,  wisdom,  and  experience 
gained  during  the  past  ten  years  or  so. 
It  will  reflect  an  appropriate  blend  of 
rigor  and  relevance,  theory,  technique, 
and  practical  applications.  It  will  pre¬ 
sent  practices  in  use  today  in  some  of 
the  excellently  managed  manufactur¬ 
ing  firms  in  America.  The  guide  will 
be  organized  around  a  generic  model 
(Figure  1)  for  performance  manage¬ 
ment.  It  will  provide  a  logical  flow  of 
the  performance  management  process. 


The  Guide 

will  reflect  an 


Chapter  2.  Visions.  What  the  De¬ 
fense  Contractor  of  the  Future  Must 
Look  Like.  Shared,  well-communi¬ 
cated  visions  of  the  future  are  critical 
to  achieving  desired  outcomes  in  the 
defense  industry.  Thus,  the  perfor¬ 
mance  management  process  logically 
begins  with  a  focus  on  visions,  guiding 
principles,  missions,  and  outcomes. 

Chapter  3.  The  Responses.  Strategy 
and  Plans  for  Continuous  Quality  and 
Productivity  Improvement .  This 
chapter  provides  an  overview  of  the 
performance  management  process.  It 
explains  the  process  and  shows  how 
the  process  can  be  a  "roadmap"  for  an 
organization’s  performance  manage¬ 
ment  efforts.  This  chapter  provides  a 
detailed  outline  of  what  follows  in  the 
guide.  This  chapter  discusses  how  to 
create  plans  for  quality  and  productiv¬ 
ity  improvement.  It  also  focuses  on  a 
critical  and  oftentimes  troublesome  ele¬ 
ment  of  performance  management,  the 
planning  process. 


Design  Criteria  for  the  Guide 

The  design  criteria,  agreed  to  by  the 
team  in  the  fall  of  1986  and  revised  in 
1987,  are  shown  below. 

—Prepare  a  guide  written  from  the 
contractor's  perspective  and  responsive 
to  contractor  and  government  needs 

— Present  a  procedure  for  designing  a 
grand  strategy  for  quality  and  produc¬ 
tivity  improvement 

— Explain  role  of  measurement  in  per¬ 
formance  improvement  efforts,  espe¬ 
cially  the  link  between  measurement 
and  incentive  methodologies 

—Identify  and  describe,  in  succinct 
form,  govemment-to-contractor  incen¬ 
tive  methodologies 

—Explain  the  Total-Factor  Productiv¬ 
ity  Measurement  Model,  Price 
Waterhouse  Cost  Definition  Method¬ 
ology,  Discounted  Cash  Flow/Shared 
Savings  Approach,  LTV  Integrated 
Productivity  Measurement  Model,  and 
other  surrogate  measurement  tools  and 
techniques 

— Document  a  technique  that  can  be 
used  to  evaluate  existing  and  proposed 
productivity  measurement  and  man¬ 
agement  systems 

— Provide  a  balanced  view  of  the  per¬ 
formance  management  process 

— Discuss  philosophy  for  making  per¬ 
formance  management  and  improve¬ 
ment  a  way  of  life 

—Test,  before  formal  publication,  the 
guide  credibility  to  government  and 
contractor  personnel. 


The  Guide 

will  reflect  an 
appropriate 
blend  of  rigor 
and  relevance, 
theory,  technique, 
and  practical 
applications. 

Chapter  1.  The  Challenger.  What 
the  Defense  Contractor  of  the  Present 
Faces.  This  chapter  concerns  the 
tremendous  stress  that  our  global, 
dynamic  economy  is  creating  on 
American  organizations.  It  explains 
that  increased  competition  is  forcing 
management  teams  to  reexamine  basic 
practices.  It  states  how  we  do  business 
must  be  scrutinized  and  modified. 


Chapter  4.  Measurement.  Theory 
and  Approaches:  How  to  Build  Qual¬ 
ity  and  Productivity  Measurement 
Systems.  This  chapter  presents  the 
theory  of  measurement  and  the 
approach  for  building  measurement 
systems.  Critically  important  philoso¬ 
phy  and  foundation  concepts  that  must 
guide  measurement  efforts  are  pro¬ 
vided.  The  distinction  between 
measurement  systems  for  control  and 
improvement  are  discussed. 

Chapter  5.  Measurement  Tech¬ 
niques.  How  to  Measure  Quality  and 
Productivity.  This  chapter  explains  the 
measurement  techniques.  Various 
models  are  presented  for  consideration. 

Chapter  6.  Improvement.  Strategies 
to  Make  Quality  and  Productivity 
Improvement  Happen.  This  chapter 
explains  how  to  improve  quality  and 
productivity.  It  provides  a  structural 
overview  of  the  state-of-the-art.  Many 
unique  techniques  are  presented  in  a 
step-by-step,  flow-process  fashion. 
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Techniques  include  recent  techniques 
used  in  performance  management 
efforts;  e.g.,  govemment-to-contractor 
incentive  methodologies  (industrial 
modernization  incentives  program), 
contractor-to-employee  incentive  pro¬ 
grams,  total  quality  management  and 
participative  management. 

Chapter  7.  Maintaining  Excellence. 
How  to  Make  Quality  and  Productiv¬ 
ity  a  Way  of  Life.  The  final  chapter  is 
devoted  to  maintaining  excellence  in 
performance  management.  It's  a  tough 
task  to  sustain  a  culture  supporting 
constant  improvement  and  innova¬ 
tion.  The  reader  should  be  challenged 
to  think  about  the  organization  of  the 
future.  Questions  like  these  will  be 
raised:  How  does  an  organization 
function  using  concepts  and  techniques 
presented  in  this  document?  How  does 
one  keep  moving  toward  a  vision 
while  simultaneously  and  continually 
updating  the  vision? 

Conclusion 

Measuring  and  managing  quality 
and  productivity  at  the  close  of  the 
1980s,  and  into  the  1990s,  will  con¬ 
tinue  to  be  complex.  This  is  partly  the 
result  of  the  complex  interrelationship 
continuing  to  exist  among  government 
entities,  competitors,  the  economy, 
technological  advances,  and  the  uncer¬ 
tain  environment.  Therefore,  imple¬ 
menting  even  the  simplest  concept  or 
technique  presented  here  will  be  a 
challenge  for  any  organization,  indus¬ 
trial  firm  or  government  entity. 

The  need  for  a  guide  is  evident.  The 
guide  presents  in  a  clear  and  succinct 
fashion  a  structured  approach  to  per¬ 
formance  management  based  on  good 
practices  being  used  successfully  in 
1988. 

Although  implementation  of  models 
and  methodologies  set  forth  in  the 
guide  will  not  ensure  success,  organi¬ 
zations  using  them  properly  will  be 
more  apt  to  succeed  in  performance 
management  than  those  who  do  not. 
We  should  recognize  that  quality  and 
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productivity  are  not  always  what  they 
seem  to  be  on  defense  system  pro¬ 
grams.  Realities  of  a  manager's  expec¬ 
tations  call  to  mind  these  lines  from 
Act  II  of  Gilbert  and  Sullivan's  HMS 
Pinafore,  which  I  had  the  pleasure  to 
see  again  recently; 

Things  are  seldom  what  they  seem, 

Skim  milk  masquerades  as  cream 

Storks  turn  out  to  be  but  logs, 

Bulls  are  but  inflated  frogs. 

Gild  the  farthing  if  you  will. 

Yet  it  is  a  farthing  still. 

Many  factors  influence  a  manager  to 
gild  the  quality  and  productivity  far¬ 
thing.  The  successful  manager  will  be 
able  to  show  it  is  possible  to  do  more 
with  less  while  using  techniques  not 
always  understood  by  others  and, 
sometimes,  without  an  incentive  for 
meaningful  improvement. 

Endnotes 

1 .  Lord  Kelvin  (William  Thomson), 
British  scientist,  supervised  the  laying 
of  the  first  successful  transatlantic 
cable  and  patented  70  inventions.  He 
received  the  title  of  Baron  Kelvin  of 
Largs  in  1892.  Because  of  his  discov¬ 
eries  about  sound,  light,  heat,  and  elec¬ 
tricity,  he  is  recognized  as  one  of  the 
greatest  scientific  intellects  of  his  time. 

2.  Riggs,  J.  L.,  and  Felix,  G.  H., 
Productivity  by  Objectives,  Engle¬ 
wood  Cliffs,  N.  J.,  Prentice-Hall,  1983. 

3.  Members  of  the  Industry  Advi¬ 
sory  Group  who  participated  in  this 
project  were  from  the  following  com¬ 
panies:  Allied  Bendix,  Bell  Helicopter, 
Boeing,  Garrett,  General  Dynamics, 
General  Electric,  Grumman,  Hercules, 
Honeywell,  Ingalls  Shipbuilding,  LTV, 
McDonnell  Douglas,  Price  Water- 
house,  Raytheon,  Rockwell  Interna¬ 
tional,  Sierracin/Sylmar,  Simmonds, 
Sundstrand,  Texas  Instruments,  TRW, 
and  Westinghouse, 


4.  Since  retirement  of  Dr.  Richard 
A.  Stimson  from  the  federal  govern¬ 
ment  in  1986,  Douglas  Reeves  (now 
retired),  Gordon  A.  Frank,  and  Kurt 
Greene,  all  from  the  Office  of  the 
Secretary  of  Defense,  have  kept  a 
watchful  eye  on  this  project. 

5 .  Report  of  the  Army  Procurement 
Research  Office  (APRO)  Number 
83-01,  subject;  "Contractor  Productiv¬ 
ity  Measurement,"  June  1984. 

6.  Report  of  the  Air  Force  Business 
Research  Management  Center 
(AFBRMC)  Number  BRMC-83-5071, 
subject:  "Development  of  a  Taxonomy 
of  Productivity  Measurement  Theories 
and  Techniques,"  February  1984. 

7.  Report  by  the  Virginia  Produc¬ 
tivity  Center  (VPC),  subject:  "The 
Study  of  Productivity  Measurement 
and  Incentive  Methodology  (Phase  Ill- 
Paper  Test),"  3  volumes,  March  1986. 

8.  Report  by  the  Virginia  Produc¬ 
tivity  Center,  subject:  "The  Study  of 
Productivity  Measurement  and  Incen¬ 
tive  Methodology  (Phase  IV-Field 
Test),"  March  1987. 

9.  Masters  of  Change:  A  Joint 
Defense-Industry  Guide  to  Quality  and 
Productivity  (Draft). 

10.  Peter  Drucker  defines  efficiency 
as  "doing  things  right"  and  effec¬ 
tiveness  as  "doing  the  right  things." 

11.  Ibid. 

12.  According  to  Peter  Drucker, 
"...In  turbulent  times,  the  first  task  of 
management  is  to  make  sure  of  the  in¬ 
stitution's  capacity  for  survival,  to 
make  sure  of  its  structural  strength  and 
soundness,  of  its  capacity  to  survive  a 
blow,  to  adapt  to  sudden  change,  and 
to  avail  itself  of  new  opportunities. . . . 
Without  productivity  objectives,  a 
business  does  not  have  direction. 
Without  productivity  measurement,  it 
does  not  have  control." 


Mr.  Acker  is  a  Professor  of  management 
at  the  Defense  Systems  Management 
College. 
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PROTOTYPES 


Cabin  Brown 


Lately,  the  term  "prototype"  has  been  used  extensive¬ 
ly  in  articles,  books  and  journals  dealing  with  the 
acquisition  of  weapon  systems  and  equipment.  However, 
the  term  is  rarely  defined  and  the  reader  is  left  to  determine 
his  or  her  perception  of  which  meaning  the  author  intended. 
This  article  is  intended  to  look  at  kinds  of  prototyping  and, 
in  the  process,  clear  up  some  confusion. 

In  the  final  report  of  the  President's  Blue  Ribbon  Com¬ 
mission  on  Defense  Management  (Packard  Commission), 
competitive  prototyping,  either  at  the  system  or  critical  sub¬ 
system  level,  was  recommended  for  all  major  weapon 
systems.  In  the  advanced  development  phase  of  a  program 
(concept  demonstration /validation  phase  of  a  major 
system),  this  prototyping  would  involve  an  informal  com¬ 
petition  of  ideas  and  technologies  rather  than  a  formal  com¬ 
petition  based  on  cost.  Early  operational  testing,  and 
developmental  testing,  should  be  performed  to  uncover 
operational  and  technical  deficiencies  before  entering  full- 
scale  development  (FSD).  At  this  stage  it  is  still  relatively 
inexpensive  to  correct  operational  and  technical  deficiencies 
uncovered  during  testing  of  the  prototypes. 


E  arly  operational  testing  and 
developmental  testing  should  be 
performed  to  uncover  operational 
and  technical  deficiencies  before 
entering  full-scale  development. 


The  Department  of  Defense  Appropriations  Act  for  fiscal 
1987  mandated  that  all  major  weapon  systems  entering  ad¬ 
vanced  development  after  September  30,  1987,  must  include 
a  competitive  prototype  strategy.  The  act  requires  that 
contracts  be  entered  into  with  at  least  two  contractors,  and 
requires  that  systems  and  subsystems  developed  undergo 
comparative  side-by-side  tests.  Contrary  to  the  Packard 
Commission  concept  of  "competitive  prototyping,"  each 
contractor  developing  a  prototype  system  or  subsystem  is 
required  to  submit  cost  estimates  for  full-scale  development 
and  production  estimates,  where  practicable,  before  testing 
begins.  When  competitive  strategy  is  not  practicable,  the 


Secretary  of  Defense  must  submit  written  notification  to  the 
Congress  explaining  that  position.  The  latest  revision  of 
Department  of  Defense  Directive  5000.1  reflects  this 
mandate. 

The  previous  discussion  reflects  the  meaning  of  "prototyp¬ 
ing"  as  defined  in  the  DSMC  Glossary  of  Defense  Acquisi¬ 
tion  Acronyms  and  Terms:  "An  original  or  model  on  which 
a  later  item  is  formed  or  based.  Usually  built  during 
Demonstration/Validation  and  tested  prior  to  Milestone  II 
decision."  This  is  normally  referred  to  as  an  "engineering" 
or  "experimental"  prototype  and  is  the  meaning  most  often 
inferred  by  the  term  "prototype." 

A  second  kind  of  prototype  is  "preproduction"  which  is 
developed  during  full-scale  development  and  represents  the 
production  configuration  in  all  aspects  of  form,  fit,  and  func¬ 
tion  except  that  it  has  not  been  fully  qualified  in  terms  of 
environmental  and  operational  testing.  A  preproduction 
prototype  is  produced  for  the  purpose  of  final  system /equip¬ 
ment/software  test  and  evaluation  before  entering  the  pro¬ 
duction  phase.  Its  purpose  is  to  verify  design  accuracy  to 
the  maximum  extent  practicable  at  this  stage  of  the  life 
cycle. 

In  software  development,  an  alternative  to  the  normal 
system  life-cycle  development  approach  is  an  evolutionary 
design  method  called  "prototyping."  Software  prototyping 
is  particularly  useful  when  requirements  are  difficult  to 
specify  in  advance,  when  requirements  may  change  signifi¬ 
cantly  during  development,  or  when  user-developer  com¬ 
munications  are  indirect  or  difficult.  It  is  usually  easier  for 
users  to  tell  what  they  do,  or  do  not,  like  about  an  existing 
prototype  than  it  is  to  express  what  they  would  like  in  an 
imaginary,  undeveloped  system.  A.  Milton  Jenkins,  Univer¬ 
sity  of  Wisconsin,  defines  prototyping  as  a  four-step  pro¬ 
cess  to:  (1)  identify  the  user's  basic  requirements,  (2)  develop 
the  initial  prototype,  (3)  let  the  user  use  the  prototype  to 
refine  requirements,  and  (4)  revise  and  enhance  the  proto¬ 
type.  Steps  (3)  and  (4)  are  normally  an  iterative  process. 
When  that  process  is  performed  early  in  advanced  develop¬ 
ment,  with  the  understanding  that  the  prototype  will  be 
discarded  after  the  requirements  are  refined  and  that  no  for¬ 
mal  documentation  will  be  delivered,  the  process  is  called 
"rapid"  prototyping. 

I  hope  this  brief  discussion  has  contributed  to  your 
understanding  of  competitive  prototyping,  engineering  and 
preproduction  prototypes,  and  rapid  prototyping.  In  a 
future  issue,  I  will  discuss  the  Navy  Rapid  Prototyping  pro¬ 
cess  for  solving  urgent  fleet  war  fighting  deficiencies. 


Mr.  Brown  is  a  Professor  of  engineering  management  in  the 
Besearch  Directorate  at  DSMC. 
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A  CAPITAL  CRISIS  IN  THE  DEFENSE 

INDUSTRY? 


(A  Summary  of  Three  Recent  Studies  of  Financial  Health  and 
Investment  Attractiveness  of  the  Industry) 

Dr.  Fred  Waelchli 


Defense  material  acquisition  in  this  country  is  a  joint  undertaking  of  three  colossal  and  diverse  national  institutions:  the  executm 
branch ,  the  legislative  branch  and  the  defense  industry.  Needs  and  interests,  institutional  and  personal,  of  participants  in  the  acquisi¬ 
tion  process  vary.  While  there  must  be  a  substantial  overlap  of  genuine  interests  among  the  three  for  the  acquisition  process  to  succeed, 
there  are  important  interests  that  inevitably  and  properly  diverge  or  conflict.  This  is  a  normal  situation,  and  it  is  the  legitimate  func¬ 
tion  of  markets  and  politics  to  resolve  these  differences. 

A  special  problem  arises,  however,  if  independent  actions  of  two  parties  combine  unexpectedly  to  threaten  serious  injury  to  the  third, 
and  thus  impair  ability  of  the  defense  acquisition  process  to  function.  Three  recent  studies  independently  suggest  this  situation  has 
happened— that  a  series  of specific  actions  taken  by  the  Department  of  Defense  and  the  Congress  between  1 984  and  1 987  have  combined 
to  severely  threaten  the  health  of  the  defense  industry.  This  article  summarizes  these  three  studies.  David  Westermann ’s  article,  ‘  ‘Govern¬ 
ment  Contract  Profit  Policy  and  Defense,  ”  in  the  May-June  issue  of  Program.  Manager,  explored  ramifications  of  problems 
highlighted  here. 

Issues  raised  are  serious.  You  should  rectgnize,  however,  that  these  studies  originated  in  the  defense  industry,  and  must  be  con¬ 
sidered,  to  some  extent,  advocacy  journalism.  No  attempt  is  made  in  this  article  to  be  even-handed;  views  expressed  are  those  of  the 
participants  in  the  studies  as  interpreted  by  the  author.  These  views  are  not  endorsed  by  the  Defense  Systems  Management  College, 
Program  Manager,  or  any  office  in  the  Department  of  Defense.  Program  Manager  solicits  thoughtful,  factual,  concise,  and 
readable  expressions  of  opposing  or  divergent  views  on  issues  raised  in  this  article. 


The  U.S.  defense  industry  is  in  genuine  trouble,  and 
its  problems  are  deepening  rapidly.  This  is  the 
ominous  theme  of  three  recent  studies  of  the  defense  in¬ 
dustry,  one  released  in  July  1987,  and  the  other  two  in 
February  1988. 

The  studies'  common  assertion  is  that  a  handful  of  un¬ 
coordinated  legislative  and  administrative  actions,  taken  in 
government  in  the  last  five  or  so  years,  has  fundamentally 
altered  the  risk/reward  relationship  in  the  defense  industry. 
These  actions  have  simultaneously  raised  defense  contrac¬ 
tors'  risks  and  lowered  their  prospective  rewards  so  that  risk- 
adjusted  returns  on  assets  invested  in  the  defense  industry 
are  no  longer  competitive  with  the  range  of  returns  available 
to  capital  invested  elsewhere.  Therefore,  we  will  see  (in  fact, 
already  see)  a  significant  capital  flight  from  the  defense  in¬ 
dustry,  with  serious  consequences  for  national  security. 

Provenance  of  Three  Studies 

The  first  study  was  by  the  Financial  Executives  Institute 
(FEI)  a  professional  organization  of  senior  financial  and  ad¬ 
ministrative  officeis  in  U.S.  business  organizations.  The 
second  study  was  by  13  senior  defense  industry  executives 
chartered  in  August  1987  as  the  ad  hoc  Defense 
Industry  Advisory  Group  (DIAG)  by  the  Defense  Industry 


and  Technology  Subcommittee  of  the  Senate  Armed  Ser¬ 
vices  Committee  (SASC).  The  third  was  commissioned  by 
three  industry  associations,  National  Security  Industrial 
Association,  Aerospace  Industries  Association,  and  Elec¬ 
tronic  Industries  Association,  and  was  performed  by  the 
MAC  group.* 

A  Mare's  Nest  of  Government  Acquisition  Actions 

Each  study  cited  the  following  governmental  actions  as 
significant  contributors  to  an  incipient  exodus  of  capital  from 
the  defense  industry.  You  should  understand  what  follows 
is,  in  each  case,  a  terse  and  necessarily  simplified  summary 
of  a  complex  action  or  situation. 

Special  Tooling  and  Test  Equipment  Investment 
Recovery.  The  Fiscal  Year  1987  Defense  Appropriation  Act 
(P.L.  99-591)  required  contractors  to  capitalize  and  amor¬ 
tize  (rather  than  expense)  costs  of  investment  in  special 
tooling  required  to  make  unique  products,  and  costs  of 
special  test  equipment.  The  FY  88-89  Defense  Authorization 
Act  (P.L.  100-180)  wrote  into  permanent  law  the  require¬ 
ment  for  contractor  investment  in  50  percent  of  special  tool¬ 
ing  costs.  Legislation  makes  payment  of  profit  of  these  in¬ 
vestments  a  matter  of  governmental  case-by-case 
discretion. 
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Reduced  Rate  of  Progress  Payments. 
The  Fiscal  Year  1987  Defense  Ap¬ 
propriation  Act  (P.L.  99-591)  reduced 
progress  payments  on  defense  con¬ 
tracts  from  80  percent  to  75  percent. 
The  earlier  rate  had  been  90  percent  or 
80  percent,  dependent  on  size  of  the 
firm. 

Restrictions  on  Completed  Contract 
Method  of  Accounting.  The  1983, 
1986,  and  1987  tax-law  revisions  suc¬ 
cessively  reduced  ability  of  the  con¬ 
tractor  to  defer  recognition  of,  and 
payment  of  taxes  on,  contract  profit 
until  completion  of  the  contract.  This 
change  results  in  earlier  tax  payments, 
with  a  consequent  negative  cash  flow 
and  a  new  need  to  finance  vanished 
working  capital  funds. 

One  Percent  Reduction  in  Profit 
Margin  Markups.  The  Fiscal  Year  1987 
Defense  Appropriation  Act  (P.L. 
99-591)  reduced  pre-negotiation  profit 
margin  markups  by  a  target  of  1  per¬ 
cent  of  costs  or  about  10  percent  of 
profits.  The  law  partially  offset  these 
reductions  with  variations  designed  to 
reward  a  contractor  for  investment  in 
facilities  and  capital  investment. 

Lower  Cost  Recoveries.  The  Con¬ 
gress  has  capped  the  amount  of  In¬ 
dependent  Research  and  Development 
(IR&D)  and  Bid  and  Proposal  (B&P) 
costs  the  contractor  can  recover 
through  overhead  billing  on  govern¬ 
ment  contracts,  and  has  prohibited  the 
payment  of  profit  on  IR&D.  The  Con¬ 
gress  added  to  the  list  of  normal  and 
unavoidable  costs  of  doing  business 
that  are  decreed  to  the  "unallowable” 
for  contract  reimbursement  pur¬ 
poses. 

Transfer  of  Rights  in  Du  iu  to 
Government.  The  Fiscal  Year  1985 
Defense  Authorization  Act  (P.L. 
98-525)  required  the  Department  of 
Defense  to  issue  regulations  defining 
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FIGURE  1.  TYPICAL  PROJECT  NET  CASH  FLOW 
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the  government's  legitimate  interests  in 
contractor  technical  data  rights.  Com¬ 
plaints  of  Department  of  Defense 
alleged  overuse  of  data  rights  provi¬ 
sion?  apparently  provoked  new  legisla¬ 
tion  (P.L.  99-961;  FY  1987  Defense 
Authorization  Act)  stating  the  congres¬ 
sional  belief  that  Department  of 
Defense  regulations  required  contrac¬ 
tors  to  give  up  too  much  proprietary 
information,  and  directing  Defense  to 
rework  regulations. 

Research  and  Development  Cost 
Sharing.  As  a  matter  of  common  prac¬ 
tice,  contractors  are  frequently  re¬ 
quired  to  pay  part  of  the  contract 
research  and  development  costs  of  new 
systems,  with  no  guarantee  of  a 
follow-on  procurement  contract  in 
which  to  recoup  the  investment. 

Fixed-Price  Development  Contracts. 
While  not  a  law  or  Department  of 
Defense  policy,  it  had  become  custom 
for  some  military  services  to  insist  on 


fixed-price  contracts  for  research  and 
development  of  new  systems,  and  to 
demand  fixed-price  production  options 
before  the  end  of  product  develop¬ 
ment.  The  effect  has  been  to  shift 
sizable,  usually  uncompensated,  risk  to 
the  contractor.  There  has  been  recent 
action  on  this  issue;  the  FY  1988 
Defense  Appropriation  Act  restricts 
Department  of  Defense  use  of  fixed- 
price  development  contracts  and  re¬ 
quires  such  contracts  to  be  approved 
by  the  Undersecretary  of  Defense  for 
Acquisition. 

SUMMARIES  OF  THREE 
REPORTS 

Financial  Executives  Institute 
Report,  The  Impact  of  Government 
Policies  on  Defense  Contractors,  July, 
1987.  This  report  uses  the  device  of  the 
cumulative  net  cash  flow  diagram  to 
illustrate  notionally  effects  of  cited 
governmental  actions.  Figure  1  shows, 
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FIGURE  2.  EFFECT  OF  GOVERNMENT  ACTIONS 
ON  PROJECT  NET  CASH  FLOW 
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for  an  actual  1970  defense  program, 
unnamed,  but  represented  as  typical, 
an  initial  gentle  rate  of  negative  cash 
flow  during  research,  development, 
and  initial  production,  which  deepens 
sharply  during  early  volume  produc¬ 
tion,  turns  up  as  cash  from  sales  ar¬ 
rives,  and  reaches  positive  territory 
(break-even)  toward  the  end  of 
production. 

Figure  2  shows  effects  the  Financial 
Executives  Institute  asserts  cited 
governmental  actions  would  have  on 
cash  flow  for  the  same  project  were  the 
project  to  start  today.  Early  negative 
cash  flow  is  deeper,  requiring  the  con¬ 
tractor  to  finance  larger  sums  for 
longer  periods,  extending  break-even 
date,  and  reducing  final  net  positive 
cash  balance.  Were  cash  flows  dis¬ 
counted,  as  is  always  the  case  in  prac¬ 
tice,  these  effects  would  be  magnified 
since  negative  cash  flow  is  early  and 
positive  cash  flow  is  late. 
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The  Financial  Executives  Institute 
report  does  not  quantify  effects  it 
describes,  but  makes  the  following 
predictions  based  on  its  analysis: 

"Participation  in  defense  contracting 
by  both  large  companies  and  smaller 
specialty  companies  will  be  reduced  as 
investment  opportunities  outside  the 
industry  become  more  attractive.  The 
defense  industrial  base  will  shrink  and 
competition  will  decline  because  the 
entry  level  investment  has  increased 
without  a  corresponding  increase  in 
the  potential  return. 

"Internationally,  the  competitive  posi¬ 
tion  of  U.S.  contractors  will  erode, 
with  a  corresponding  negative  impact 
on  the  U.S.  balance  of  payments. 

"Finally,  but  most  importantly,  the 
worldwide  technology  lead  of  the  U.S. 
defense  industry  may  not  be 
maintained." 


Defense  Industry  Advisory 
Group  Report  to  Subcommittee  on 
Defense  Industry  and  Technology, 
Senate  Armed  Services  Committee, 
February  5,  1988.  The  report  is  the 
broadest,  dealing  with  acquisition 
issues  beyond  the  eight  governmental 
actions  cited  here,  but  also  addressing 
those  actions.  The  Defense  Industry 
Advisory  Group  was  formed  in 
August  1987  by  Senator  Jeff 
Bingaman,  chairman,  and  Senator  Phil 
Gramm,  ranking  minority  member,  to 
advise  the  Senate  Armed  Services 
Committee  Defense  Industry  and 
Technology  Subcommittee  on 
". .  .those  aspects  of  the  acquisition  pro¬ 
cess  that  stifle  innovation,  drain  good 
talent  away  from  the  acquisition 
system,  and  threaten  our  technological 
lead." 

Chairman  John  Rittenhouse, 
General  Electric  Aerospace  Corpora¬ 
tion,  and  the  panel  identified  20  issues 
in  three  thematic  areas  which  they 
believed  adversely  affected  ability  of 
the  current  defense  acquisition  process 
to  provide  for  needs  of  the  armed 
forces,  and  its  ability  to  ensure  con¬ 
tinued  advancement  of  technology  and 
industrial  productivity  necessary  for 
national  security. 

The  first  Defense  Industry  Advisory 
Group  theme  was  "People:  the  Pro¬ 
curement  Workforce,  and  Organiza¬ 
tional  Relationships  within  DOD" 
(two  issues);  second,  "Process: 
Streamlining  and  Stabilizing  the  Pro¬ 
cess,  Including  Issues  of  Contractor 
Financing  and  Investment"  (14  issues); 
finally,  "Trust:  the  Relationships 
among  Congress,  the  Executive 
Branch,  and  Industry"  (four  issues). 
The  first  four  problem  statements  of 
the  "process  issues"  center  on  the  cited 
governmental  actions  and  raise  the 
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problem  of  capital  flight  from  the  in¬ 
dustry.  Those  four  issues  are  stated  as 
follows  in  the  Defense  Industry  Ad¬ 
visory  Group  study: 

"Issue  1:  The  Conflict  Between  Profit 
and  Investment  Policies.  The  issue  is 
that  government  acquisition  policy 
seeks  to  create  incentives  for  contrac¬ 
tors  to  make  investments  which  benefit 
the  government  and  to  provide  reason¬ 
able  profits  from  which  such  in¬ 
vestments  can  be  made.  This  policy 
became  confused  over  the  last  two 
years  as  a  result  of  conflicting 
legislative  and  regulatory  initiatives. 
Without  resolution  of  the  conflict,  this 
policy  will  fail. 

"Issue  2:  Profits  and  Costs.  Financial 
policies  enacted  in  recent  years  to 
reduce  DOD  outlays  and  the  federal 
deficit  will  increase  procurement  costs 
and  reduce  national  security  in  the 
long  term.  Profits  account  for  less  than 
10%  of  defense  expenditures,  while 
contract  costs  account  for  more  than 
90% .  Many  of  these  costs  are  the  result 
of  unnecessary  government  regula¬ 
tions.  Attention  should  focus  on  non¬ 
productive  costs. 

"Issue  3:  Government  Policy  on  In¬ 
dependent  Research  and  Development 
(IR&D).  For  the  past  several  years. 
Congress  has  established  ceilings  on 
Government-allowable,  Independent 
Research  and  Development  and  Bid 
and  Proposal  (B&P)  costs.  These 
ceilings  inhibit  industry  investment  in 
advanced  technologies  and  new 
products. 

"This  year.  Congress  will  consider 
whether  to  alter  or  eliminate  the 
Government's  traditional  support  for 
independent  research  and  develop¬ 
ment.  Because  IR&D  is  at  the  founda¬ 
tion  of  the  technological  superiority  of 
our  armed  forces,  the  IR&D  system 
should  be  strengthened  as  an  invest¬ 
ment  in  the  future. 

"Issue  4:  Shifting  Undue  Risk  to  the 
Contractor.  Disregarding  the  lessons 
learned  from  failures  of  similar  pro¬ 
curement  methods  in  the  past,  the 
DOD  is  now  employing  procurement 
methods  which  shift  unmeasurable 
risks  to  contractors  in  three  different 
ways.  First,  contractors  are  being  re- 
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quired  to  pay  a  portion,  sometimes 
substantial,  of  the  development  costs 
of  Defense  Department  systems,  under 
a  practice  called  'cost  sharing'  . 
Second,  contractors  are  being  required 
to  enter  into  fixed-price  contracts  early 
in  development,  when  the  uncertain¬ 
ty  is  so  substantial  that  it  is  virtually 
impossible  to  know  the  precise  costs  of 
new  systems.  Third,  contractors  are 
being  asked' to  provide  the  Defense 


The  domestic 
defense  industry  is 
no  longer  an  attrac¬ 
tive  investment; 
reward  no  longer 
compensates  ade¬ 
quately  for  risk,  and 
capital  is  fleeing. 
Strength  in  the 
defense  industry  is 
migrating  offshore. 


Department  with  priced  production 
options  before  full-scale  development 
has  begun....  All  of  these  requirements 
shift  undue  risk  to  the  contractor, 
drain  industry  resources  from  in¬ 
vestments  in  technology  and  produc¬ 
tivity,  and  will  ultimately  affect  our 
nation's  ability  to  maintain 
technological  superiority." 

The  Defense  Industry  Advisory 
Group  Report  poses  three  "...fun¬ 
damental  questions  of  how  DOD 
should  do  business,"  derived  from  con¬ 
sideration  of  the  cited  set  of  govern¬ 
ment  actions: 

"The  most  essential  business  equation, 
in  which  measurable  risk  is  balanced 
by  profit  potential,  no  longer  describes 


defense  business.  How  can  this  balance 
be  restored? 

"Current  policies  discourage  invest¬ 
ment  in  technology  and  productivity. 
How  should  they  be  changed? 

"Small  manufacturing  firms  are  forced 
to  pay  a  high  price  for  participating  in 
the  government  market  by  having  to 
give  up  the  fruits  of  their  innovation. 
What  effect  does  this  have  on  competi¬ 
tion?" 

The  Defense  Industry  Advisory 
Group  Study  is  qualitative;  like  the  FEI 
study,  it  does  not  attempt  to  quantify 
the  effects  it  describes.  It  asserts  keys 
to  improving  the  acquisition  process 
are  "...streamlining  and  program 
stability."  35. 

The  MAC  Report,  The  Impact  on 
Defense  Industrial  Capability  of 
Changes  in  Procurement  and  Tax 
Policy,  1984-1987,  dated  February, 
1988.  This  report  is  the  only  quantified 
analysis  among  the  three  studies.  It 
takes  the  cumulative  net  cash-flow 
methodology  used  in  the  Financial  Ex¬ 
ecutive  Institute  Report  and  applies  it, 
with  new  procurement  rules,  to  actual 
data  from  nine  historical  Department 
of  Defense  acquisition  programs,  all 
but  one  profitable,  and  each  managed 
by  a  different  defense  contractor.  The 
intent  was  to  estimate  effects  of 
governmental  actions  on  contractor 
cash  flow,  now  and  in  the  future.  Like 
the  Financial  Executive  Institute 
Report,  the  idea  was  to  estimate  finan¬ 
cial  impacts  on  past  programs  if  the 
new  rules  had  been  in  effect.  Among 
their  findings: 

"The  return  on  investment  on  the  pro¬ 
grams  analyzed  would  have  been  less 
than  the  return  necessary  to  preserve 
shareholder  value.  Put  simply,  there 
would  have  been  no  financial  reason 
to  bid  the  programs. 

"Profits  on  current  work  will  be 
substantially  reduced— by  an  average 
of  23%  on  the  companies'  defense 
business. 

"Companies  will  be  forced  to  borrow 
heavily,  but  the  additional  financing 
required  will,  for  some  companies, 
likely  exceed  the  amount  that  can  be 
borrowed. 
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“As  companies  feel  the  squeeze  on 
available  capital,  they  will  be  forced 
to: 

— Reduce  (already  declining)  con¬ 
tractor-funded  research  and  develop¬ 
ment  investment, 

— Reduce  risk  by  opting  for  low- 
technology  alternatives. 

— Reduce  investment  needed  for 
productivity  enhancement  and 
modernization. 

— Decline  to  bid  programs  with  un¬ 
realistic  provisions. 

"While  the  full  financial  impact  may 
not  be  reflected  in  the  industry's  finan¬ 
cial  statements  for  three,  or  even  more, 
years,  the  industry  is  already  be¬ 
ginning  to  respond  in  these  terms.  The 
financial  impact  will  probably  show 
first  in  subcontractor  financial 
statements. 

"The  governmental  changes  will  result 
in  a  less  efficient  industry.  Cutting 
back  on  capital  investment  is  mort¬ 
gaging  the  future,  slowing  down  the  in¬ 
dustry's  ability  to  improve  production 
efficiency. 

“Competition  will  be  reduced  because 
there  will  be  fewer  competitors  willing 
to  bid  on  future  programs." 


Dismal  Message 

The  composite  message  of  three 
studies  is  dismaying.  The  domestic 
defense  industry  is  no  longer  an  attrac¬ 
tive  investment;  reward  no  longer 
compensates  adequately  for  risk,  and 
capital  is  fleeing.  Strength  in  the 
defense  industry  is  migrating  offshore. 
If  the  present  trend  goes  unchecked, 
the  U.S.  defense  industry  will  shrink, 
competition  will  decline,  technological 
progress  will  stagnate,  and  the  quest 
for  preeminent  performance  in  U.S. 
weapon  systems  will  falter  as  industry 
declines  to  accept  the  prerequisite 
development  risks. 

No  one  planned  for  such  a  situation, 
and  nobody  wants  it.  Why  is  it  now 
a  real  prospect? 

Program  Manager 


The  Predicament 

If  we  accept  the  validity  of  the  three 
studies,  the  mistake  leading  into  the 
current  predicament  was  a  series  of 
nearly  simultaneous,  uncoordinated 
actions  by  the  Congress  and  the 
Department  of  Defense,  each  logical¬ 
ly  designed  for  a  limited  objective,  but 
whose  sum  foreshadows  a  possible  na¬ 
tional  security  nightmare. 


JLhe  way  out, 
in  direct  contrast  to 
the  way  in,  must  be 
carefully  and 
thoughtfully 
planned  and 
coordinated  by 
those  with  power 
to  make  changes, 
the  Department  of 
Defense  and  the 
Congress. 


The  first  lesson,  therefore,  is  that  ac¬ 
quisition  is,  or  takes  place  within,  a 
complex  social  system.  Complex  social 
systems  can  be  controlled  only  by  well- 
designed  and  properly  crafted  control 
and  administrative  devices;  "simple" 
control  devices  do  not  control  and  in¬ 
evitably  produce  unanticipated, 
usually  unwelcome,  side  effects. 

The  way  out,  in  direct  contrast  to 
the  way  in,  must  be  carefully  and 
thoughtfully  planned  and  coordinated 
by  those  with  power  to  make  changes, 
the  Department  of  Defense  and  the 
Congress.  Proposed  actions,  singly 
and  in  combination,  must  be  tested  for 
impact  on  the  system  before  they  are 
imposed.  The  MAC  Report 
argues: 


“...Congress  should  ensure  that  the 
type  of  uncoordinated  policy  change 
that  led  to  this  unintended  utcome  is 
not  repeated.  The  Congress  and  DOD 
must  undertake  some  form  of  impact 
assessment. .  .before  introducing 
policies  whose  cumulative  impact  on 
the  industry  is  as  severe  as  we  have 
shown. 

“...Congress  and  DOD  should  develop 
a  clearer  vision  of  the  type  of  defense 
industry  that  will  be  required  in  five 
to  ten  years.  It  should  test  its  procure¬ 
ment  policies  against  these  re¬ 
quirements,  and  ensure  that  they  pro¬ 
vide  an  incentive  to  invest  in  the 
production  facilities  and  equipment, 
research,  and  human  resources  that  the 
industry  and  the  DOD  will 
require." 

In  discussing  the  Defense  Industry 
Advisory  Group  Report  with  the 
Senate  Armed  Services  Committee 
staff,  a  Defense  Systems  Management 
College  faculty  team  proposed  com¬ 
missioning  a  major  study  to  examine 
legitimate  needs  and  interests  of  the  Ex¬ 
ecutive  branch,  the  Congress  and  in¬ 
dustry  to  see  if  a  viable,  “win-win," 
modus  vivendi,  supportive  of  national 
security  and  acceptable  to  all 
stakeholders  (some  used  the  word 
"treaty"),  could  be  devised. 

Clearly,  there  are  different  ways  to 
approach  this  problem.  Surely  we  need 
to  start  now  to  construct  a  reasoned 
plan  to  ensure  that  we  will  never  again 
find  ourselves  unwitting  victims  of 
fragmented,  uncoordinated,  contradic¬ 
tory,  and  ultimately  damaging  acquisi¬ 
tion  policy  and  regulations. 

As  this  article  is  written,  there  are 
signs  the  capital  flight  problem  is 
becoming  visible  to  the  Congress  and 
the  Department  of  Defense.  Senator 
Bingaman's  subcommittee  on  the 
defense  industrial  base  heard  testimony 
on  two  of  the  three  reports  I  have 
reviewed.  The  Department  of  Defense 
appears  to  be  backing  off  on  fixed- 
price  development  contracts  and 
shows  evidence  of  second  thoughts  on 
cost  sharing.  By  now,  there  may  be 
other  changes  affecting  the  capital- 
flight  problem. 

As  yet,  however,  there  is  no  sign  of 
a  coordinated,  thoughtful,  strategic  ap- 

J  uly-  August  1988 


•  • 

v  V.V.V.- 


tsm 


-  ’J 

,■» ->V‘ 
'•/oVoV-j 

m 


sg 

§ 

S3 

Wm 


:»v 


proach  toward  defining  or  crafting  the 
optimum  or  desired  relationship  be¬ 
tween  government  and  the  defense  in¬ 
dustry.  There  is  every  evidence  that  we 
are  proceeding  to  "piecemeal"  our  way 
out  of  the  predicament  just  as  we 
"piecemealed"  our  way  in.  Thus,  we 
guarantee  no  reasoned  or  long-term 
systemic  solution  to  the  structural 
problem. 

Specific  Approaches  to 
Capital-Flight  Crisis 

What  can  be  done  to  resolve  today's 
apparent  defense  industry  capital  flight 
crisis?  Here  is  a  portion  of  the  MAC 
Report's  response: 

"Our  study  makes  explicit  an  issue  that 
DOD  procurement  policy  must  face. 
Business  is  fundamentally  about  risks 
and  returns.  As  essentially  the  only 
purchaser  of  highly  specialized  defense 
equipment,  DOD  controls  both  sides 
of  the  risk/return  balance  (at  least  for 
major  systems  procurement).  In  the 
period  we  have  examined,  DOD  and 
the  Congress  decided  to  adjust  what 
was  viewed  as  an  imbalanced  risk- 
return  relationship.  Unfortunately 
[they)  reduced  rewards  and  increased 
risks  simultaneously,  with  not  one,  but 
multiple,  uncoordinated  adjustments. 
At  the  same  time.  Congress  signifi¬ 
cantly  increased  the  industry's  capital 
requirements  (by  reducing  progress 
payments  and  deferred  tax 
financing). 

"While  some  in  DOD  now  claim  the 
industry  is  much  more  like  commercial 
industry.  Wall  Street  is  saying  it  will 
not  provide  it  with  capital  at  the  same 
rate  as  commercial  industry.  Wall 
Street  might  provide  the  capital  if  it 


saw  the  opportunity  for  high  returns 
(as  it  does  for  biotechnology  com¬ 
panies,  for  example);  the  industry 
might  live  with  low  profits  if  the 
government  provided  more  of  the 
financing  and  did  away  with  cost  shar¬ 
ing,  fixed-price  development,  and 
other  unreasonable  risks.  But,  as  mat¬ 
ters  currently  stand,  government  has 
stepped  out.  Wall  Street  is  unwilling  to 
step  in,  and  the  industry  is  unable 
to. 

"Solutions  have  to  address  both  prof- 
fitability  and  risk,  which  have  to  be  in 
balance.  This  introduces  the  question 
of  what  is  the  appropriate 
balance?" 

What  is  the  appropriate  and 
politically  acceptable  balance  between 
profitability  and  risk  in  the  defense  in¬ 
dustry?  That  seems  to  be  the  question 
to  answer  soon  in  a  coordinated  way 
by  everyone  concerned. 

Bottom  Line  for  Industry 

Individual  governmental  actions  in 
the  three  reports  are  complicated,  and 
contributions  of  each  action  to  the 
general  problem  are  difficult  to  assess, 
as  each  study  makes  clear.  The  overall 
issue,  however,  could  not  be  simpler. 

As  a  nation,  we  choose  to  acquire 
military  material  and  equipment 
through  a  privately  financed,  private¬ 
ly  managed,  free,  and  non-coercive 
market  system.  In  this  system,  pri¬ 
vately  owned  assets  are  invested  to  ob¬ 
tain  a  return,  and  these  assets  flow  in¬ 
exorably  to  investments  appearing  to 
offer  the  best  risk-adjusted  returns.  An 
investment  offering  a  less-than- 
competitive,  risk-adjusted  return  will 
not  attract  capital. 


The  common  theme  of  the  three 
studies  is  the  effort  of  the  government, 
in  the  eight  issue  areas  and  elsewhere, 
to  chip  away  at,  to  limit,  and  to  lower 
the  risk-adjusted  return  allowed  on 
private  assets  invested  in  defense  in¬ 
dustry,  hoping  to  lower  weapon  ac¬ 
quisition  costs.  If  these  actions  com¬ 
bine  to  force  the  aggregate  risk- 
adjusted  return  on  defense  work  below 
the  competitive  range,  as  some  argue 
is  already  the  case,  the  result  must  be 
the  flight  of  capital,  new  and  replace¬ 
ment,  from  the  defense  industry — 
surely  an  intolerable  prospect  for  na¬ 
tional  security. 

Copies  of  the  studies  discussed  in 
this  article  may  be  obtained  as  follows: 

—FE1  Report:  Financial  Executives 
Institute,  Washington  Office, 
1100  Seventeenth  Street,  N.W., 
Washington,  D.C.  20036.  67. 

— D1AG  Report:  Subcommittee  on 
Defense  Industry  and  Technology, 
Senate  Committee  on  Armed  Services, 
United  States  Senate,  Washington, 
D.C.  20510.  68. 

— MAC  Report:  National  Security  In¬ 
dustrial  Association,  National  Head¬ 
quarters,  1015  15th  Street,  N.W.,  Suite 
901,  Washington  D.C.  20005.  69. 

‘The  MAC  Group  is  a  consulting  firm 
formed  by  a  group  of  Harvard 
Business  School  professors. 


Doctor  Woe  kbit  is  a  Professor  of  manage¬ 
ment,  Business  Management  Department , 
at  the  Defense  Systems  Management  Col¬ 
lege  and  a  member  of  the  DSMC  Center 
for ■  Acquisition  Management  Policy. 
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PROGRAM  MANAGER’S  GUIDE 

FOR  MONITORING  CONTRACTOR 
OVERHEAD/INDIRECT  COST 

Ronald  L.  Baker 


L 

JL  IS! 


II  n  December  1984,  the  Deputy  Secretary  of  Defense 

ML  issued  a  memorandum,  "Initiative  to  Reduce 
Overhead."  The  memorandum  identified  ten  principles  of 
overhead  cost  control  and  was  forwarded  to  chief  executive 
officers  of  major  defense  firms  seeking  their  help  in  con¬ 
trolling  overhead  costs.  These  principles  gave  government 
and  industry  objectives  to  monitor  and  manage  overhead 
cost  and  were  intended  to  set  the  direction  and  tone  of  the 
Department  of  Defense  (DOD)  commitment  to  overhead 
reduction.  The  ten  principles  of  overhead  cost  control  ap¬ 
peared  in  the  March-April  1985  issue  of  Program 
Manager. 

Since  then,  government  and  industry  have  done  much  to 
focus  on  reducing  overhead/indirect  costs  on  weapon 
systems.  Developing  improved  overhead  cost-reduction  pro¬ 
cedures  is  increasingly  important  because  overhead  is  a  ma¬ 
jor  percentage  of  total  contractual  costs.  Increased 
mechanization  produces  an  increasing  overhead  percentage 
in  the  absence  of  strong  control  measure.  A  major  reason 
for  this  increase  is  the  requirement  for  highly  skilled  per¬ 
sonnel  to  maintain  computer  hardware  and  software.  Pro¬ 
grams  such  as  the  Industrial  Modernization  Incentives  Pro¬ 
gram  (IMIP)  reduce  direct  cost  elements  through  capital  in¬ 
vestment.  Without  overhead  reduction,  portions  of  the 
potential  total  cost  reduction  will  be  lost. 

Through  the  IMIP,  the  DOD  provides  contractors  incen¬ 
tives  to  invest  in  capital  improvements.  This  improves  pro¬ 
ductivity  and  reduces  costs.  "Productivity  Improvements  to 
Reduce  Contractor  Overhead,"  a  research  paper  completed 
in  April  1987  by  an  Air  Force  Reserve  group  assigned  to  the 
Aeronautical  Systems  Division,  indicated  many  techniques 
and  concepts  used  in  the  IMIP  program  to  identify  changes 
and  provide  incentives  to  contractors  have  direct  applica¬ 
tion  to  overhead  reductions.  Indirect  or  overhead  costs  are 
always  significant  or  dominant  in  the  cost  of  systems  ac¬ 
quisition.  These  costs  are  allocated  to  all  programs  in  a  con¬ 
tractor's  plant.  This  makes  it  difficult  for  program  managers 
to  apply  control  mechanisms.  This  multiple-program  aspect 
of  overhead  is  similar  to  contractors'  capital  facilit'  -<•  and 
equipment  which  normally  benefit  their  programs,  direct 
program  costs  and  indirect  overhead  costs  share  most  ex¬ 
penses;  i.e.,  labor,  equipment,  and  materials.  As  such,  in¬ 
vestments  could  reduce  direct  and  overhead  costs  and  im¬ 
prove  productivity. 

Currently,  overhead  cost  is  controlled  by  the  Forward 
Pricing  Rate  Agreement  (FPRA).  This  utilizes  provisional 
billing  rates  based  on  historical  experience  and  estimates  of 
business  volume  during  the  period  involved.  Final  rates  are 
negotiated  after  the  end  of  the  period  and  an  audit  of  ac¬ 
tual  overhead  costs.  The  FPRA  approach  has  been  in  place 
for  the  past  15  years.  One  study  states  it  "emphasizes  three 
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factors:  the  concept  of  cost  avoidance,  the  Air  Force  assump¬ 
tion  of  responsibility  for  overhead  management  at  plants 
for  which  it  has  responsibility,  and  a  larger  role  for  the  on¬ 
site  plant  office  in  the  overhead  process."  Cost  avoidance 
should  focus  efforts  on  future  cost  reduction.  Also,  the  Air 
Force,  rather  than  the  Defense  Contract  Audit  Agency, 
monitors  and  settles  overhead  matters  and  is  involved  in 
the  on-site  Air  Force  office  in  surveillance  and  settlement 
activities. 

The  current  Air  Force  system  for  overhead  control  uses 
FPRAs  primarily,  but  includes  the  Monitoring  Overhead 
through  Discrete  Evaluations  (MODE)  computer  program; 
overhead  should-cost  evaluations;  the  Air  Force  indirect  cost 
monitoring  team  consisting  of  an  overhead  monitor,  an  in¬ 
dustrial  engineer  and  an  auditor;  and  the  proposed  use  of 
traditional  fee-sharing  to  control  overhead  expenditures.  A 
recent  approach  is  the  use  of  Memoranda  of  Understanding 
(MOU)  between  the  Air  Force  and  contractors  covering 
some  voluntary  measures  that  the  contractors  will  take  to 
reduce  overhead. 

Recently,  the  Defense  Systems  Management  College 
developed  the  Program  Managers  Guide  for  Monitoring 
Contractor  Overhead/ Indirect  Cost.  This  was  written  for 
the  non-financial  manager  and  provides  a  succinct  tool  for 
program  managers  and  acquisition  staffs  to  use  in  their  over¬ 
sight  roles  of  weapon  systems  overhead  costs.  The  guide  in¬ 
cludes  overhead  cost  topics  such  as:  the  definition  of 
overhead  and  indirect  cost,  creation  of  indirect  cost  pools, 
compliance  with  cost  accounting  standards,  and  explana¬ 
tion  of  procedures  and  tools  used  by  the  indirect  cost  team 
in  monitoring  contractors.  There  is  no  single  source  of 
literature  available  to  provide  acquisition  personnel  with  a 
complete  overview  of  indirect/overhead  costs.  The  guide 
provides  this  overview  and  gives  the  program  manager  a 
greater  understanding  on  the  role  and  responsibility  of 
monitoring  program  costs. 

This  guide  will  be  available  early  in  FY  89. 


Mr.  Baker,  formerly  a  member  of  the  Research  Staff at  the  Defense 
Systems  Management  College,  is  now  Deputy  Pregram  Manager 
for  Training  Systems,  U.S.  Marine  Corps  Research,  Development 
and  Acquisition  Command,  Quantico,  Va. 
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PROCUREMENT 

REFORM 

INITIATIVES: 

COMPETITION 


The  Honorable  Bill  Nichols 


II  spoke  to  an  Army  group  of  ac- 
JIL.  quisition  leaders  in  July  1985,  on 
the  importance  attached  by  the  Con¬ 
gress  to  the  Competition  in  Contract¬ 
ing  Act  (CICA).  Before  addressing 
more  recent  procurement  reform  ini¬ 
tiatives,  a  word  about  competition. 

In  1982,  there  were  skeptics  within 
the  Department  of  Defense  thinking 
competition  would  cause  a  host  of  hor¬ 
rors;  those  outside  the  Department 
believed  nothing  could  stop  it  from 
buying  from  favorite  suppliers.  So  far, 
time  has  proved  the  skeptics 
wrong. 

Procurement  actions  awarded  com¬ 
petitively  have  increased  dramatically, 
from  48  percent  in  1983  to  88  percent 
in  1987.  The  dollar  value  of  com¬ 
petitively  awarded  contracts  increased 
from  41.2  percent  in  1983  to  58.8  per¬ 
cent  in  1987. 

The  vigil  is  not  over.  It  is  too  soon 
to  say  everyone  within  the  Department 
is  responsible  for  acquisition  — 
contracting  officer,  program  manager, 
and  those  involved  in  requirements 
generation — necessarily  believe  com¬ 
petition  is  a  good  thing.  That  change 
in  mind-set  won't  happen  overnight.  It 
won't  occur  unless  the  drive  for  com¬ 
petition  is  done  in  a  sensible  manner. 
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As  I  said  in  1985,  soon  after  the 
Competition  in  Contracting  Act 
passed,  the  Congress  is  looking  for'  a 
process  in  which  people  act  as  if  money 
being  spent  is  their  own— and  that  they 
use  common  sense  when  spending  that 
money.  That  means  the  person  con¬ 
sider  all  available  information  and 
make  a  sound  business  judgment. 

One  important  aspect  of  sound 
business  judgment  is  whether  to  utilize 
competition.  Then,  we  believed  peo¬ 
ple  were  not  looking  at  benefits  of 
competition,  like  reduced  price,  better 
quality,  etc.,  because  the  system 
fostered  an  environment  penalizing 
one  for  pursuing  competition.  It  was 
safer  and  less  time-consuming  to  ex¬ 
ecute  a  sole-source  contract  than  to  go 
competitive. 

By  legislating  changes  in  that  pro¬ 
cess,  I  hope  we  removed  some  im¬ 
pediments.  We  created  a  presumption 
that  competition  made  sense.  We  did 
not  abolish,  however,  the  need  to 
make  an  informed  judgment  con¬ 
cerning  the  best  acquisition 
approach. 

The  fact  is  that  virtually  every  pro¬ 
curement  is  different  in  some  respect. 
No  rule  or  regulation  can  be  drafted  to 
apply  neatly  and  give  the  best  result  in 
every  situation.  So,  no  matter  how 


Corifprssman  Bill  Nichols 
(D  ,  Ala.) 

many  rules  there  are,  only  you  can 
assess  the  situation  and  structure  the 
best  acquisition  strategy  for  the 
government. 

We  are  looking  for  personal  ac¬ 
countability  and  elimination  of  un¬ 
necessary  layers  of  bureaucracy,  not  to 
have  a  person  to  point  the  finger  at 
when  things  go  wrong;  rather,  to  give 
authority  back  to  the  person  respon¬ 
sible  for  making  a  decision,  and  to  stop 
second-guessing. 

This  concern  was  the  genesis  for  the 
program  executive  officer  (PEO)  con¬ 
cept.  Although  this  concept  was  con¬ 
ceived  by  the  Packard  Commission, 
the  Congress  was  receptive  to  the  idea 
because  it  fit  hand-in-glove  with 
another  concern  of  ours— that  there  is 
no  accountability  in  the  Department  of 
Defense  acquisition  process.  Program 
managers  were  changing  frequently, 
and  so  many  people  were  involved  in 
making  programmatic  decisions  that  it 
was  impossible  to  say  any  one  person 
was  in  charge  of  a  program. 

By  embracing  the  program  executive 
officer  concept  we  are  trying  to 
streamline  the  process  by  establishing 
simple  lines  of  communication  and 
responsibility.  We  hoped  this  might 
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force  a  reevaluation  of  the  necessity  for 
reviewing  organizations.  Even  if  it  does 
not  result  in  a  reduction  in  the 
bureaucracy,  however,  we  hope  it  will 
force  people  to  make  inputs  to  the 
decision-maker  at  the  start,  rather  than 
second-guessing  decisions  already 
made. 

Some  said  various  procurement 
reform  measures  take  away  authority 
of  the  contracting  officer  or  program 
manager.  I  agree  in  some  cases,  when 
the  Congress  believed  decisions  being 
made  were  not  appropriate,  it  has 
acted  to  limit  that  exercise  of  judg¬ 
ment.  A  good  example  is  Navy  fixed- 
price  contracts  for  research  and 
development.  We  understood  what 
they  were  trying  to  accomplish  but  we 
did  not  believe  using  fixed-price  con¬ 
tracts  was  the  appropriate  way  to  meet 
the  objective.  When  this  policy  con¬ 
tinued  despite  objections,  the  Congress 
adopted  a  prohibition  in  the  defense 
segment  of  the  continuing  resolution  to 
use  fixed-price  contracts  for  research 
and  development. 

Wfile  the  Congress  has  legislated 
many  prohibitions  on  some  acquisition 


No  rule  or 
regulation  can  be 
drafted  to  apply 
neatly  and  give  the 
best  result  in  every 
situation. 


activities,  these  prohibitions  are,  in 
reality,  simple  boundaries  on  the  range 
of  options  available  to  the  acquistion 
manager;  prohibitions  are  in  no  way, 
shape,  or  form  intended  as  a  substitute 
for  ability  to  examine  facts  and  cir¬ 
cumstances  and  judge  what  is  the  best 
action.  If  you  examine  statements 
made  by  members  of  the  Congress  in 
the  last  few  years,  it  should  become  ap¬ 
parent  the  consistent  underlying  theme 
is  that  people  cannot  and  should  not 
hide  behind  rules  and  regulations.  The 
system  cannot  work  otherwise.  We 
need  to  make  decisions  based  on 
uni  ie  circumstances  of  each  procure¬ 
ment.  The  system  relies  on  each  per¬ 
son  making  the  best  decisions  about 
each  aspect  of  the  acquisition 
process. 

The  Congress  knows  the  system  is 
as  good  as  the  people  who  run  it;  thus, 
their  concern  about  quality  of  the 
work  force.  In  the  last  few  years,  there 
have  been  proposals  to  improve  the  lot 
of  acquisition  personnel.  While  I  don't 
necessarily  always  agree  with  the  ap¬ 
proach  taken,  you  should  know  what 
these  proposals  entail. 

They  include  proposals  to  establish 
an  elite  acquisition  corps,  establish 
minimum  educational  criteria  for  ac¬ 
quisition  personnel  and,  finally, 
eliminate  established  pay  grades  in 
favor  of  a  system  recognizing  out¬ 
standing  work  with  bonuses  and  merit 
pay. 

The  Congress  emphasized  educa¬ 
tional  opportunities  for  government 


JL  he  Congress 
knows  the  system  is 
as  good  as  the  people 
who  run  it;  thus, 
their  concern  about 
quality  of  the  work 
force. 


acquisition  personnel.  In  1987,  we 
mandated  that  Department  of  Defense 
reassess  acquisition  training  programs 
conducted  by  the  military  services  and 
the  Defense  Logistics  Agency.  We 
established  a  requirement  that  training 
of  acquisition  personnel  at  least  be 
coordinated  within  the  Department  of 
Defense. 

Some  proposals  are  controversial. 
We  may  not  have  all  the  answers,  but 
we  are  concerned. 

I  hope  you  know  the  Congress 
recognizes  there  must  be  some  judg¬ 
ment  involved  in  the  acquisition  pro¬ 
cess.  While  we  put  boundaries  on  ex¬ 
ercising  that  judgment,  we  have  at¬ 
tempted  to  make  sure  you  retain  the 
ability  to  make  necessary  decisions. 
Remember  what  an  old  Texan  once 
said:  "There's  nothing  in  the  middle  of 
the  road  but  yellow  stripes  and  dead 
armadillos." 


This  is  adapted  from  remarks  made  at  the 
Defense  Systems  Management  College  by 
The  Honorable  Bill  Nichols ,  Democratic 
Congressman  from  Alabama ,  to  the  Army 
Competition  Leaders  Conference. 
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USE  OF  PRODUCTION  RATE  AS 
INDEPENDENT  VARIABLES 
IN  COST  MODELS 

Dr.  Michael  N.  Beltmmo 
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An  AIM-7  Sea  Sparrow  Missile  is  fired  from  the  aft  Basic  Point  Defense  Missile  System  Jetuncher  on  the  Amphibious  Assault  Ship  USS  I  wo  Jima,  I.PH-2. 
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I  here  is  a  growing  trend  in  cost  analysis  to  incorporate 
-JL.  production  rate  variables  into  cost  estimating  rela¬ 
tionships  intended  for  general  application.  An  article  in  The 
Journal  of  Cost  Analysis  (Camm,  Gulledge  and  Womer, 
"Production  Rate  and  Contractor  Behavior,"  Vol.  5,  No. 
1)  identifies  key  studies  related  to  the  effect  of  production 
rate  on  cost  and  provides  an  excellent  quantitative  analysis. 
These  notes  were  written  from  a  different  perspective; 
specifically,  hypotheses  should  not  be  accepted  unless  they 
can  withstand  theoretical  and  logical  scrutiny  as  well  as  em¬ 
pirical  validation. 

The  following  comments  suggest  logical  hurdles  that  need 
to  be  cleared  to  accept  a  model  that  relates  production  cost 
to  production  rate  for  similar  items  produced  in  different 
environments. 

A  conceptual  overview  of  production  rate  and  its  pre¬ 
sumed  effect  on  production  cost  gets  to  the  heart  of  the  mat¬ 
ter.  There  is  general  agreement  that  producing  at  an  optimal 
rate  results  in  the  lowest  cost,  all  else  being  equal.  This 
allows  available  capital  and  labor,  direct  and  indirect,  to 
be  fully  utilized.  At  a  lower  rate,  surplus  is  charged  to  the 
smaller  output.  At  a  higher  rate,  inefficient  substitutions 
(e.g.,  added  labor  instead  of  additional  equipment,  or  in¬ 
adequate  maintenance)  are  made  and  increase  cost. 
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There  are  many  possible  ways  to  produce  a  particular  item 
by  making  capital/labor  and  raw  material /purchased  part 
tradeoffs,  for  example.  Each  firm  has  unique  characteristics 
driving  its  production  processes.  In  particular,  two  general 
types  of  plants  exist  in  defense  industries.  One  is  product 
oriented  where  each  output  has  a  dedicated  production  line 
including  labor,  tooling,  and  capital  equipment.  In  many 
cases,  a  product-oriented  plant  produces  a  single  item.  The 
other  is  process  oriented  where  a  particular  function  like 
circuit-board  production  or  testing  is  performed  for  end- 
items.  The  following  assumptions  may  be  made  about  op¬ 
timal  production  rates  and  plant  types: 

—A  discrete,  optimal  production  rate  may  exist  for  a  specific 
item  in  a  product-oriented  plant. 

—Two  product-oriented  plants  may  have  very  different 
optimal-production  rates  for  the  same  product  depending 
upon  how  they  are  facilitized,  staffed,  and  tooled. 

—An  optimal  production  rate  for  a  process-oriented  plant 
depends  on  total  through-put;  that  is,  a  decrease  in  produc¬ 
tion  rate  for  one  item  may  be  compensated  for  by  an  in¬ 
crease  for  another  with  no  decline  in  efficiency  (i.e. ,  increase 
in  cost). 
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TABLE  1.  SPARROW  AIM-7F  BUY  VS. 
PRODUCTION  QUANTITIES 


FY 

RAYTHEON 

GENERAL  DYNAMICS 

Buy 

Production 

Buy 

Production 

1972 

100* 

1973/74 

225 

24 

15* 

— 

1975 

600 

167 

70 

— 

1976 

880 

332 

210 

— 

1977 

1110 

925 

210 

70 

1978 

1400 

1086 

750 

395 

1979 

900 

1077 

1310 

391 

1980 

1144 

1000 

300 

1027 

|  1981 

— 

722 

— 

961 

1982 

— 

881 

— 

— 

1983 

~ 

41 

Total 

6359 

6255“ 

2865 

2844** 

‘Not  included  in  production  quantities.  Note:  First  lots  may  be  develop¬ 
ment  units. 

“For  Raytheon,  the  difference  between  the  total  buy  and  the  total  pro¬ 
duction  quantity  is  because  the  production  quantity  does  not  include  the 
first  buy  of  100  units  and  the  FY  73/74  and  FY78  production  data  sup¬ 
plied  by  NAVAIR  were  each  missing  two  units. 

“For  General  Dynamics,  the  difference  between  the  total  buy  and  the 
total  production  quantity  is  because  the  production  quantity  does  not  in¬ 
clude  the  first  buy  of  15  units  and  the  FY78  and  FY80  production  data 
supplied  by  NAVAIR  were  in  net  error  of  minus  six  units. 


The  typical  production  cost  vs.  pro¬ 
duction  '•ate  relationship  is  charac¬ 
terized  by  a  U-shaped  curve. 
Therefore,  a  given  rate  may  be  too  low 
for  one  plant  to  operate  efficiently  but 
too  high  for  another.  This  is  different 
than  learning-curve  theory  which 
holds  that  "more  is  cheaper."  Thus,  the 
idea  of  correlating  quantities  by  time 
period  with  cost  for  similar  but  not 
identical  items  produced  in  a  variety 
of  environments  is  difficult  to  fathom 
in  the  absence  of  other  information. 

Even  if  the  serious  logical  issues  are 
ignored,  there  are  practical  problems 
related  to  calculating  the  effect  of  pro¬ 
duction  rate  on  cost.  The  first  problem 
is  that  rate  data  are  not  reported 
routinely.  Therefore,  annual  procure¬ 
ment  quantities  are  commonly  used  as 
a  proxy  for  production  rate  because 
they  are  easier  to  obtain  than  data 
related  to  annual  deliveries.  Even 
deliveries  may  misrepresent  when  pro- 
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duction  actually  occurred  as  prime 
contractor  work  may  be  completed  but 
not  delivered  pending  readiness  of  the 
government  user  to  accept  it  or  prime 
contractor  labor  could  be  essentially 
complete  with  delivery  delayed  pend¬ 
ing  the  integration  of  government  fur¬ 
nished  equipment;  e.g.,  missile 
warheads  or  aircraft  engines. 

The  two  may  vary  significantly. 
Consider  annual  procurement  quan¬ 
tities  and  annual  delivery  quantities 
for  Sparrow  AIM-7F  shown  in  Table 
1.  Both  firms  used  the  time  lag  between 
contract  award  and  the  beginning  of 
production  to  achieve,  a  relatively 
stable  production  ratt.  Dramatic  rate 
variances  implied  by  annual  procure¬ 
ment  quantities  that  could  have  in¬ 
creased  costs  were  avoided;  therefore, 
any  production  rate  effects  calculated 
for  that  program  based  upon  procure¬ 
ment  quantities,  rather  than  deliveries, 
may  be  more  apparent  than  real. 


Another  problem  observed  when 
analysts  apply  production  rate  factors 
is  that  they  do  not  adjust  learning- 
curve  slopes.  This  causes  double 
counting.  Production  rate  effects  are 
the  effect  of  quantity  per  time  period 
on  cost  and  learning  is  the  effect  of 
quantity  on  cost;  therefore,  learning 
curves  implicitly  include  production 
rate  effects,  although  they  may  be 
cancelled  when  they  fluctuate 
significantly  during  the  period 
covered.  The  need  to  adjust  learning- 
curve  slopes  when  applying  a  rate 
slope  is  obvious.  An  appropriate 
method  for  making  the  adjustment  has 
not  been  identified. 

There  is  widespread  agreement  con¬ 
cerning  the  importance  of  production 
rate  as  a  cost  driver  in  the  abstract.  It 
is  possible  to  develop  cost  estimating 
relationships  which  include  an  ap¬ 
propriate  production  rate  variable  for 
individual  plants  or  specific  products 
in  a  particular  plant.  It  has  not  been 
demonstrated  that  various  production 
rates  for  similar  products  in  different 
production  environments  have,  or 
should  have,  the  same  effect  on  cost. 


The  author  is  president  of  Beltmmo  and 
Associates ,  a  deftnse  acquisition  consulting 
firm  in  Los  Angeles. 


July-August  1988 


.  •  • 

•  k  Km.  Km  *  M  1 

■  >  *  kV  * 


•  • 

.’-V*  LV---"  v 


m 


(Swf 


•  • 

.V.-'.VA 


mk 


TOTAL  QUALITY  MANAGEMENT: 
A  DOD  EXAMPLE 

Commander  J.  C.  Boudreaux,  USN 


Quality  management 
principles  are  not 
new  in  the  United  States. 
Walter  Shewhart  wrote 
about  control  of  industrial 
quality  as  early  as  1931. 
Dr.  W.  Edwards  Deming 
and  Dr.  J.  M.  Juran,  two 
prominent  American  quality 
theorists,  introduced  quality 
management  techniques  to 
Japan  in  the  1950s.  S.  V. 
Feigenbaum,  P.  B.  Crosby 
and  others  have  translated 
general  theories  into  prac- 


Fourth,  quality  cannot  be 
inspected  into  a  product.  In¬ 
spection  identifies  the  defec¬ 
tive  product  but  cannot 
eliminate  cause  of  the  defect. 

Fifth,  statistical  and 
graphic  techniques  support 
the  quality  management  de¬ 
cision-making  process.  Sim¬ 
ple  graphs  define  process 
capability  and  can  be  used 
by  the  manager  to  make  de¬ 
cisions  concerning  changes 
to  the  process. 


tical  applications  that  are  changing  the  corporate  outlook  Sixth,  all  levels  of  the  organization  must  be  trained 


of  United  States  industry  toward  quality  management .  to  understand  the  company's  quality  philosophy  and  to  im- 


Although  each  individual  quality  theorist  has  a  slightly  dif¬ 
ferent  emphasis,  there  are  six  basic  principles  to  which  all 
adhere. 

First,  management,  not  the  worker,  is  responsible  for  pro¬ 
cess  quality.  Management  must  provide  resources  and  policy 
decisions  to  make  process  changes  and  facilitate  tasks  of  the 
work  force. 

Second,  continuous  improvement  of  process  quality 
through  control  of  variability  yields  productivity  improve¬ 
ment.  Quality  management  emphasizes  continuous  search 
for  a  better  productivity  by  improving  work  methods  and 
reducing  rework. 

Third,  the  worker  can  provide  a  significant  input  to  pro¬ 
cess  quality  control.  The  operator  of  a  process  is  in  the  best 
position  to  identify  process  weaknesses  and  to  offer  sugges¬ 
tions  for  process  improvement. 
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plement  that  policy.  Every  level  of  an  organization  must 
have  the  common  mission  and  common  language  with 
which  to  communicate. 

Resurgence  of  these  principles  has  been  stimulated  by 
American  industry's  reaction  to  the  Japanese  competition 
in  international  markets.  United  States  industry  has  begun 
using  these  total  quality  management  principles  to  regain 
the  competitive  position  in  the  world  market.  At  least  one 
group  of  government  agencies  is  following  industry's  lead. 

A  recent  example  of  total  quality  management  illustrates 
applicability  of  these  quality  principles  to  production  and 
administrative  areas  within  Department  of  Defense  organiza¬ 
tions.  The  Naval  Aviation  Depot  (NADEP),  North  Island, 
San  Diego,  Calif.,  is  a  Department  of  Defense  organization 
making  a  long-term  commitment  to  total  quality 
management. 
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The  NADEP  is  a  complex  industrial 
facility  performing  depot-level 
maintenance,  overhauling  and  re¬ 
pairing  engines  and  components  from 
Naval  aircraft,  and  performing  similar 
maintenance  on  support  systems 
related  to  naval  aviation.  The  Navy 
operates  six  NADEPs  at  Norfolk,  Va. , 
Cherry  Point,  N.C.,  Jacksonville,  Fla., 
Pensacola,  Fla.,  Alameda,  Calif.,  and 
San  Diego,  Calif.  The  six  NADEPs 
represent  a  complex  mix  of  products 
and  processes. 

One  early  effort  to  improve  quality 
control  was  an  artisan  certification 
program  managed  by  the  Quality 
Assurance  Department  at  each 
NADEP.  Previous  programs  included 
quality  circles,  quality  goals,  slogan 
programs,  quality  reporting  and 
auditing  activities.  These  programs 
contributed  to  quality  awareness  but 
fell  short  of  providing  coordinated, 
continuing  improvement  in  the  overall 
quality  of  produi  t  and  in  the  efficiency 
of  NADEP  operations. 

By  instituting  quality  management 
techniques,  NADEP  at  North  Island 
focused  on  increasing  productivity  and 
providing  a  product  to  meet  customer 
expectations.  The  NADEP  emphasized 
long-term  management  commitment 
to  change  corporate  attitude. 
Previously,  management  approached 
change  with  fear  and  uncertainty. 
After  total  quality  management  prac¬ 
tices  were  implemented,  management 
actively  sought  and  encouraged 
change. 

Using  quality  management,  the 
organization  was  trained  at  all  levels 
to  think  in  terms  of  process  quality. 
The  work  force  was  taught  to  use  sim¬ 
ple  graphical  methods  of  statistical 
process  control  to  make  statistically 
based  decisions  affecting  their  pro¬ 
cesses.  The  new  attitude  included 
mutual  action  among  departments  of 
the  organization  for  process  quality. 
Customer  requirements  were  used  to 
define  good  quality  and  the  definition 
of  customer  was  extended  to  include 
the  user  of  products  within  and  outside 
the  organization.  One  management 
role  in  quality  management  was  to 
assist  each  organization  element  to 
recognize  internal  and  external 
customers  and  requirements. 
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The  quality  management  program  at 
NADEP  in  North  Island  evolved 
slowly  during  4  years.  Initial  activity 
included  exchanging  books  and 
discussing  current  industrial  trends  in 
quality  management.  Interest  grew 
and,  after  visits  to  local  industry,  a 
small  group  of  managers  started  an  in¬ 
formal  training  program  based  on 
seminars  of  Dr.  Deming.  Between 
1980-84  the  awareness  and  training  ef¬ 
fort  exposed  200  upper  and  middle 
managers  to  quality  management 
ideas.  A  network  developed  to  obtain 
and  distribute  literature  concerning 
quality  activity  occurring  in  private  in¬ 
dustry.  Dr.  Deming  provided  personal 
support  to  the  network  in  the  form  of 
appearances  at  North  Isiand  to  discuss 
quality  theory. 

In  1984,  the  commanding  officer  of 
NADEP,  North  Island,  commissioned 
a  demonstration  effort  in  the  Manufac¬ 
turing  Division  of  the  facility.  The 
demonstration  attempted  to  establish 
implementation  procedures  and  apply 
quality  management  techniques  to  the 
working  environment.  Specific  objec¬ 
tive  was  to  establish  the  customer-to- 
supplier  relationship  between  the 
grinding  and  plating  shops  to  reduce 
defects  and  reduce  cost  of  correcting 
defects.  By  identifying  and  breaking 
down  communication  barriers  between 
shops,  the  NADEP  management 
started  a  continuous  cycle  of  improve¬ 
ment  in  the  processes  supported  by  the 
two  shops.  The  resulting  cooperation 
between  the  shops  was  directed  at 
building  quality  into  the  processes  and 
eliminating  the  requirement  for  exten¬ 
sive  inspection  for  defects.  The  initial 
training  in  group  decision-making 
established  common  methods  to  ex¬ 
amine  the  process.  Communications 
were  sustained  at  regular  meetings  be¬ 
tween  the  shops. 

In  addition  to  working-level 
meetings,  shop  supervisors  met  week¬ 
ly  with  the  next  level  of  supervision. 
This  management  team  established 
priorities  and  assigned  resources  to 
working-level  activities.  This  quality 
process  was  developed  using  guidelines 
from  Kaoru  Ishakawa's  Guide  to 
Quality  Control  but  was  uniquely 
adapted  by  the  Naval  Aviation  Depot 
to  its  organization. 
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Any  quality  improvement  system  is 
a  process  and  evolves  by  using  tech¬ 
niques  of  quality  management.  The 
NADEP  quality  improvement  system 
was  systematically  changed  over  a 
period  of  years.  This  dynamic  manage¬ 
ment  effort  was  successful  in  trans¬ 
forming  the  daily  business  routine 
from  conflict  to  teamwork.  In  one 
specific  case,  combined  efforts  of  the 
plating  and  grinding  teams  reduced  the 
defect  rate  on  a  H-46  helicopter  blade 
fold  pin  rework  from  70  to  3  percent. 
Detailed  techniques  used  to  identify 
and  eliminate  contributors  to  the  defect 
rate  were  part  of  the  statistical  process 
control  methods  of  quality  manage¬ 
ment. 

While  the  NADEP  demonstration  ef¬ 
fort  continued  in  selected  shops,  upper 
management  was  taking  plant-wide  in¬ 
itiatives,  using  lessons  learned  from  the 
demonstration.  From  the  demonstra¬ 
tion  effort,  management  realized  that 
principles  of  process  quality  manage¬ 
ment  applied  to  service  and  support 
processes  as  well  as  to  production  line 
processes.  Concepts  were  successfully 
applied  to  such  diverse  areas  as  long- 
range  planning  activities  and  ad¬ 
ministration  of  individual  travel 
procedures. 

The  customer-supplier  emphasis  for 
service-oriented  efforts  were  similar  to 
the  production  effort.  Each  manage¬ 
ment  group  asked  initial  questions: 
What  is  our  product?  Who  is  our 
customer?  Who  are  our  suppliers? 

With  answers  to  these  questions  the 
management  used  basic  process  quality 
management  techniques  to  measure 
performance  of  the  process  that  they 
managed.  Management  identified 
elements  of  the  process  they  used  to 
transform  suppliers'  inputs  into  the 
output  for  their  customers.  Each  group 
identified  and  prioritized  opportunities 
to  improve  their  process  and  to 
enhance  the  quality  of  their  ultimate 
product. 

Success  of  the  improvement  initially 
depended  on  energetic  communica¬ 
tion.  Communicating  with  suppliers 
ensured  input  specifications  were  met; 
communicating  with  process  operators 
ensured  the  quality  definitions  support 
the  ultimate  product;  and  com¬ 
municating  with  the  customer  ensured 
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the  product  would  meet  the  customer's 
expectations.  Functions  of  "continuous 
process  audit"  and  "control  point 
sampling"  were  carried  out  contin¬ 
uously  by  management.  Process  audit 
activity  was  initiated  by  management 
using  information  from  the  customer 
or  from  workers  that  indicated  an 
opportunity  to  improve  the  system. 
Control  point  sampling  allowed 
management  to  monitor  process  fluc¬ 
tuations  using  measurements  estab¬ 
lished  and  collected  by  the  process 
operators  at  the  working  level. 

Once  the  process  control  system  was 
in  place,  management  continuously 
asked  questions:  How  can  we  reduce 
variability  in  our  product  quality  by 
reducing  variability  in  our  control 
points?  Have  conditions  of  our  sup¬ 
plier's  inputs  changed?  Have 
customers'  requirements  changed? 
Translating  the  answers  to  these  ques¬ 
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tions  into  action  brought  about 
systematic  changes  to  improve  process 
quality. 

Principles  demonstrated  at  North 
Island  are  applicable  everywhere  in  the 
Department  of  Defense.  By  using  this 
process  control  system,  management 
can  more  closely  control  quality  of 
output  and  avoid  the  quick-reaction 
crisis  that  leads  to  inefficiency.  Advan¬ 
tages  gained  through  quality  control 
ultimately  result  in  increased  produc¬ 
tivity  and  reduced  cost  through 
elimination  of  process  inefficiencies. 
The  process  control  system  applies 
regardless  of  the  nature  of  the  product. 
The  concepts  apply  to  all  systems  in 
the  Department  of  Defense  whether  the 
product  is  hardware,  software,  paper¬ 
work  or  services.  Ideas  of  customer-to- 
supplier  interaction  have  tremendous 
potential  for  the  Department  of 
Defense. 


Just  as  United  States  industry  is 
being  driven  by  competitive  pressure 
to  reexamine  management  techniques, 
the  Department  of  Defense  is  being 
driven  by  budget  reduction  to  improve 
productivity  and  increase  cost  effec¬ 
tiveness.  The  Department  of  Defense 
has  taken  specific  actions  to  address 
specific  productivity  and  cost- 
effectiveness  symptoms. 

Total  quality  management 
represents  a  proved  method  that  can 
be  applied  in  all  Department  of 
Defense  organizations  to  increase  pro¬ 
ductivity  and  to  reduce  cost.  Budget 
pressures  demand  action.  Inaction  will 
ultimately  degrade  the  effectiveness  of 
our  defense  system. 


Commander  Boudreaux  is  assigned 
to  the  Naval  Air  Systems  Command, 
Washington,  D.C. 
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